[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:41 dist-test-slave-z7j8 krb5kdc[5439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:41 dist-test-slave-z7j8 krb5kdc[5439](info): set up 2 sockets
Sep 26 20:45:41 dist-test-slave-z7j8 krb5kdc[5439](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 20:45:42.559309 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:45:42 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:45:42.744307 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46261
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:43.080127 5452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:43.080637 5452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:43.080821 5452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:43.088640 5452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:43.088812 5452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:43.088914 5452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:43.089016 5452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:43.089098 5452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:43.099288 5452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:46261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:43.100695 5452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:43.102094 5452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:43.114872 5457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:43.114902 5460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:43.114835 5458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:43.115502 5452 server_base.cc:1034] running on GCE node
I20260926 20:45:43.116353 5452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:43.117923 5452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:43.119187 5452 hybrid_clock.cc:648] HybridClock initialized: now 1790455543119136 us; error 47 us; skew 500 ppm
I20260926 20:45:43.119588 5452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:43 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455543, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:43.122954 5452 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:45:43.124657 5452 webserver.cc:460] Webserver started at http://127.5.78.62:40171/ using document root <none> and password file <none>
I20260926 20:45:43.125429 5452 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:43.125593 5452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:43.125932 5452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:43.129534 5452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "6088ecdeabc84de1ae5218914e4bd1b5"
format_stamp: "Formatted at 2026-09-26 20:45:43 on dist-test-slave-z7j8"
server_key: "6df67451f505dacc528e11dbe5d69953"
server_key_iv: "63dfc2a5019dd9dc11c6603e9ce046dc"
server_key_version: "encryptionkey@0"
I20260926 20:45:43.130497 5452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "6088ecdeabc84de1ae5218914e4bd1b5"
format_stamp: "Formatted at 2026-09-26 20:45:43 on dist-test-slave-z7j8"
server_key: "6df67451f505dacc528e11dbe5d69953"
server_key_iv: "63dfc2a5019dd9dc11c6603e9ce046dc"
server_key_version: "encryptionkey@0"
I20260926 20:45:43.136101 5452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 20:45:43.140065 5467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:43.141852 5452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:45:43.142091 5452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "6088ecdeabc84de1ae5218914e4bd1b5"
format_stamp: "Formatted at 2026-09-26 20:45:43 on dist-test-slave-z7j8"
server_key: "6df67451f505dacc528e11dbe5d69953"
server_key_iv: "63dfc2a5019dd9dc11c6603e9ce046dc"
server_key_version: "encryptionkey@0"
I20260926 20:45:43.142341 5452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:43.163769 5452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:43.168486 5452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:43.168802 5452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:43.188433 5452 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:46261
I20260926 20:45:43.188467 5518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:46261 every 8 connection(s)
I20260926 20:45:43.189931 5452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:43.192698 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 5452
I20260926 20:45:43.192992 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:45:43.194029 5519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:43.194970 5432 external_mini_cluster.cc:1428] Setting key 47dc5e7bdf2ff0e678a43bf1cffcb379
I20260926 20:45:43.205533 5519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5: Bootstrap starting.
I20260926 20:45:43.208770 5519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:43.209921 5519 log.cc:824] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:43.212775 5519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5: No bootstrap required, opened a new log
I20260926 20:45:43.220181 5519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46261 } }
I20260926 20:45:43.220561 5519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:43.220672 5519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6088ecdeabc84de1ae5218914e4bd1b5, State: Initialized, Role: FOLLOWER
I20260926 20:45:43.221295 5519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46261 } }
Sep 26 20:45:43 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:43.221536 5519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:43.221684 5519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:43.221864 5519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:43.224776 5519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46261 } }
I20260926 20:45:43.225343 5519 leader_election.cc:302] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6088ecdeabc84de1ae5218914e4bd1b5; no voters:
I20260926 20:45:43.226063 5519 leader_election.cc:288] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:43.226186 5524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:43.227365 5524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6088ecdeabc84de1ae5218914e4bd1b5, State: Running, Role: LEADER
I20260926 20:45:43.227993 5524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46261 } }
I20260926 20:45:43.228385 5519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:43.230763 5526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6088ecdeabc84de1ae5218914e4bd1b5. Latest consensus state: current_term: 1 leader_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46261 } } }
I20260926 20:45:43.231150 5526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:43.231737 5525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6088ecdeabc84de1ae5218914e4bd1b5" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46261 } } }
I20260926 20:45:43.232039 5525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:43.236028 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:43.198732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.5.78.62:46261)
0926 20:45:43.199607 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:43.199629 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:43.200642 (+ 1013us) server_negotiation.cc:407] Connection header received
0926 20:45:43.209444 (+ 8802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:43.209469 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:43.209880 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:43.210343 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:43.213317 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:43.214707 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:43.216311 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:43.216991 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:43.217212 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:43.223206 (+ 5994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:43.223215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:43.223229 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:43.223267 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:43.227244 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:43.228453 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:43.228461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:43.228469 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:43.228693 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:43.229274 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:43.229279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:43.229281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:43.230089 (+ 808us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:43.233382 (+ 3293us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:43.233739 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 20:45:43.234268 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":369,"threads_started":1}
I20260926 20:45:43.238068 5529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:43.244235 5529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:43.253536 5529 catalog_manager.cc:1347] Generated new cluster ID: ccf3638352fa4f4489cbc352467007a8
I20260926 20:45:43.253688 5529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:43.272709 5529 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:43.274746 5529 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:43.296042 5529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6088ecdeabc84de1ae5218914e4bd1b5: Generated new TSK 0
I20260926 20:45:43.296965 5529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:45:43.350585 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46261
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:43.718286 5545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:43.718636 5545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:43.718747 5545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:43.726652 5545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:43.726778 5545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:43.726895 5545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:45:43.737025 5545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:43.738534 5545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:43.739867 5545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:43.753098 5550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:43.753453 5551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:43.753463 5553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:43.754288 5545 server_base.cc:1034] running on GCE node
I20260926 20:45:43.755007 5545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:43.755964 5545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:43.757222 5545 hybrid_clock.cc:648] HybridClock initialized: now 1790455543757144 us; error 69 us; skew 500 ppm
I20260926 20:45:43.757578 5545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:43 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455543, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:43.760804 5545 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:45:43.762291 5545 webserver.cc:460] Webserver started at http://127.5.78.1:33883/ using document root <none> and password file <none>
I20260926 20:45:43.763056 5545 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:43.763201 5545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:43.763515 5545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:43.766721 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "0f4a05102ef745fc80290f7d62fbdd39"
format_stamp: "Formatted at 2026-09-26 20:45:43 on dist-test-slave-z7j8"
server_key: "20f28b05a6449d2ea970ee7861636a0f"
server_key_iv: "48e7785f2cf2a60754f3122f0fb46901"
server_key_version: "encryptionkey@0"
I20260926 20:45:43.767589 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "0f4a05102ef745fc80290f7d62fbdd39"
format_stamp: "Formatted at 2026-09-26 20:45:43 on dist-test-slave-z7j8"
server_key: "20f28b05a6449d2ea970ee7861636a0f"
server_key_iv: "48e7785f2cf2a60754f3122f0fb46901"
server_key_version: "encryptionkey@0"
I20260926 20:45:43.773100 5545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:43.777029 5560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:43.778682 5545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:45:43.778885 5545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "0f4a05102ef745fc80290f7d62fbdd39"
format_stamp: "Formatted at 2026-09-26 20:45:43 on dist-test-slave-z7j8"
server_key: "20f28b05a6449d2ea970ee7861636a0f"
server_key_iv: "48e7785f2cf2a60754f3122f0fb46901"
server_key_version: "encryptionkey@0"
I20260926 20:45:43.779116 5545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:43.802742 5545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:43.806974 5545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:43.807298 5545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:43.808252 5545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:45:43.809571 5545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:43.809682 5545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:43.809827 5545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:43.809898 5545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:43.843834 5545 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:46693
I20260926 20:45:43.843892 5672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:46693 every 8 connection(s)
I20260926 20:45:43.845180 5545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:45:43.845616 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 5545
I20260926 20:45:43.845875 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:45:43.846643 5432 external_mini_cluster.cc:1428] Setting key 0ad8a12f8c6eb704835ac4524b494025
Sep 26 20:45:43 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455543, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:45:43.867411 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:43.848483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:38435 (local address 127.5.78.62:46261)
0926 20:45:43.848927 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:43.848935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:43.850182 (+ 1247us) server_negotiation.cc:407] Connection header received
0926 20:45:43.853173 (+ 2991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:43.853177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:43.853242 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:43.853345 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:43.855169 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:43.855852 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:43.857355 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:43.857780 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:43.857920 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:43.861685 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:43.861690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:43.861693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:43.861725 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:43.864215 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:43.865067 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:43.865073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:43.865076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:43.865184 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:43.865667 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:43.865671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:43.865673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:43.865916 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:43.866026 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:43.866825 (+ 799us) server_negotiation.cc:299] Negotiation successful
0926 20:45:43.867069 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":222,"threads_started":1}
I20260926 20:45:43.868463 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:43.849090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46261 (local address 127.5.78.1:38435)
0926 20:45:43.849979 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:43.850050 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:43.852852 (+ 2802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:43.853638 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:43.853653 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:43.854338 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:43.854991 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:43.855009 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:43.855984 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:43.855987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:43.857196 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:43.857206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:43.858006 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:43.858013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:43.858263 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:43.859017 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:43.859039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:43.861441 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:43.864356 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:43.864368 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:43.864385 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:43.864944 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:43.865315 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:43.865320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:43.865323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:43.865557 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:43.866077 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:43.866084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:43.866457 (+ 373us) client_negotiation.cc:769] Sending connection context
0926 20:45:43.866718 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 20:45:43.867197 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":345,"threads_started":1}
I20260926 20:45:43.870040 5673 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46261
I20260926 20:45:43.870354 5673 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:43.871079 5673 heartbeater.cc:511] Master 127.5.78.62:46261 requested a full tablet report, sending...
I20260926 20:45:43.873291 5484 ts_manager.cc:194] Registered new tserver with Master: 0f4a05102ef745fc80290f7d62fbdd39 (127.5.78.1:46693)
I20260926 20:45:43.874851 5484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:38435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:45:43.890520 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46261
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:44.222734 5679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:44.223039 5679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:44.223214 5679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:44.230572 5679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:44.230706 5679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:44.230892 5679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:45:44.240592 5679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:44.241882 5679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:44.243263 5679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:44.256822 5684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:44.256891 5685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:44.256924 5687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:44.257725 5679 server_base.cc:1034] running on GCE node
I20260926 20:45:44.258378 5679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:44.259397 5679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:44.260636 5679 hybrid_clock.cc:648] HybridClock initialized: now 1790455544260575 us; error 54 us; skew 500 ppm
I20260926 20:45:44.260968 5679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:44.264178 5679 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:45:44.265584 5679 webserver.cc:460] Webserver started at http://127.5.78.2:46783/ using document root <none> and password file <none>
I20260926 20:45:44.266240 5679 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:44.266335 5679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:44.266731 5679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:44.269541 5679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "b24d6263f65144dd905bcea44c54e39b"
format_stamp: "Formatted at 2026-09-26 20:45:44 on dist-test-slave-z7j8"
server_key: "625deb52646b8bbdfa501c8d0f1a6b30"
server_key_iv: "a3fab53b58261bfd901e8743a4114b8c"
server_key_version: "encryptionkey@0"
I20260926 20:45:44.270310 5679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "b24d6263f65144dd905bcea44c54e39b"
format_stamp: "Formatted at 2026-09-26 20:45:44 on dist-test-slave-z7j8"
server_key: "625deb52646b8bbdfa501c8d0f1a6b30"
server_key_iv: "a3fab53b58261bfd901e8743a4114b8c"
server_key_version: "encryptionkey@0"
I20260926 20:45:44.275218 5679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:45:44.278753 5694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:44.280007 5679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:45:44.280169 5679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "b24d6263f65144dd905bcea44c54e39b"
format_stamp: "Formatted at 2026-09-26 20:45:44 on dist-test-slave-z7j8"
server_key: "625deb52646b8bbdfa501c8d0f1a6b30"
server_key_iv: "a3fab53b58261bfd901e8743a4114b8c"
server_key_version: "encryptionkey@0"
I20260926 20:45:44.280393 5679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:44.295347 5679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:44.299614 5679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:44.299911 5679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:44.300812 5679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:45:44.302153 5679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:44.302225 5679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:44.302374 5679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:44.302500 5679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:45:44.334776 5679 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:42785
I20260926 20:45:44.334816 5806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:42785 every 8 connection(s)
I20260926 20:45:44.336052 5679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:45:44.338143 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 5679
I20260926 20:45:44.338330 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:45:44.338886 5432 external_mini_cluster.cc:1428] Setting key 4877c1784e41a197d07a36a72530411a
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:45:44.358063 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.339602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:49351 (local address 127.5.78.62:46261)
0926 20:45:44.339769 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:44.339775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:44.340451 (+ 676us) server_negotiation.cc:407] Connection header received
0926 20:45:44.342445 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:44.342455 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:44.342546 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:44.342658 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:44.344112 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.345140 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.346908 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.347432 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.347568 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.351885 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:44.351890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:44.351893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:44.351927 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:44.354734 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.355661 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.355665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.355668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.355825 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.356346 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.356350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.356352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.356627 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:44.356748 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:44.357482 (+ 734us) server_negotiation.cc:299] Negotiation successful
0926 20:45:44.357711 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 20:45:44.359184 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.339381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46261 (local address 127.5.78.2:49351)
0926 20:45:44.340280 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:44.340334 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:44.342143 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:44.342865 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:44.342873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:44.343370 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:44.343930 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:44.343947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.345302 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.345308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:44.346757 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:44.346770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.347682 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.347690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:44.347925 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.348687 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:44.348710 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:44.351567 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:44.354908 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:44.354923 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:44.354948 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:44.355506 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:44.356006 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:44.356010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:44.356012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:44.356240 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:44.356736 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:44.356742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:44.357081 (+ 339us) client_negotiation.cc:769] Sending connection context
0926 20:45:44.357340 (+ 259us) client_negotiation.cc:240] Negotiation successful
0926 20:45:44.357939 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":371,"threads_started":1}
I20260926 20:45:44.360895 5807 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46261
I20260926 20:45:44.361340 5807 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:44.362141 5807 heartbeater.cc:511] Master 127.5.78.62:46261 requested a full tablet report, sending...
I20260926 20:45:44.363811 5484 ts_manager.cc:194] Registered new tserver with Master: b24d6263f65144dd905bcea44c54e39b (127.5.78.2:42785)
I20260926 20:45:44.364630 5484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:49351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:45:44.381767 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46261
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:44.701591 5812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:44.701933 5812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:44.702107 5812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:44.709518 5812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:44.709682 5812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:44.709872 5812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:45:44.719835 5812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46261
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:44.721149 5812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:44.722528 5812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:44.737231 5817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:44.737203 5818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:44.738125 5812 server_base.cc:1034] running on GCE node
W20260926 20:45:44.738489 5820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:44.739125 5812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:44.740099 5812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:44.741328 5812 hybrid_clock.cc:648] HybridClock initialized: now 1790455544741266 us; error 93 us; skew 500 ppm
I20260926 20:45:44.741648 5812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:44.744865 5812 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:45:44.746342 5812 webserver.cc:460] Webserver started at http://127.5.78.3:44747/ using document root <none> and password file <none>
I20260926 20:45:44.747072 5812 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:44.747277 5812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:44.747586 5812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:44.750452 5812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "f9d9586376a441a380575c76234e33dc"
format_stamp: "Formatted at 2026-09-26 20:45:44 on dist-test-slave-z7j8"
server_key: "16cbe3329ab95ff81556316d7b5cc6f8"
server_key_iv: "0ce80422530f39dfa08f1bd37cbeadc1"
server_key_version: "encryptionkey@0"
I20260926 20:45:44.751201 5812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "f9d9586376a441a380575c76234e33dc"
format_stamp: "Formatted at 2026-09-26 20:45:44 on dist-test-slave-z7j8"
server_key: "16cbe3329ab95ff81556316d7b5cc6f8"
server_key_iv: "0ce80422530f39dfa08f1bd37cbeadc1"
server_key_version: "encryptionkey@0"
I20260926 20:45:44.756119 5812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:44.759405 5827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:44.760699 5812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:44.760864 5812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "f9d9586376a441a380575c76234e33dc"
format_stamp: "Formatted at 2026-09-26 20:45:44 on dist-test-slave-z7j8"
server_key: "16cbe3329ab95ff81556316d7b5cc6f8"
server_key_iv: "0ce80422530f39dfa08f1bd37cbeadc1"
server_key_version: "encryptionkey@0"
I20260926 20:45:44.761045 5812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:44.780905 5812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:44.785529 5812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:44.785871 5812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:44.786891 5812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:45:44.788349 5812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:44.788422 5812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:44.788571 5812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:44.788631 5812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:44.821534 5812 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:46035
I20260926 20:45:44.821584 5939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:46035 every 8 connection(s)
I20260926 20:45:44.822890 5812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:45:44.828850 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 5812
I20260926 20:45:44.829064 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:45:44.829573 5432 external_mini_cluster.cc:1428] Setting key 3ce1c918b09375d23f7c1b475176ecd2
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:44.844131 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.825529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:37515 (local address 127.5.78.62:46261)
0926 20:45:44.825719 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:44.825723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:44.826984 (+ 1261us) server_negotiation.cc:407] Connection header received
0926 20:45:44.828806 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:44.828812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:44.828900 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:44.829002 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:44.831026 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.831852 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.834096 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.834582 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.834684 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.838286 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:44.838291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:44.838294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:44.838325 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:44.840951 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.841650 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.841655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.841657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.841785 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.842193 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.842197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.842199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.842558 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:44.842688 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:44.843453 (+ 765us) server_negotiation.cc:299] Negotiation successful
0926 20:45:44.843696 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:45:44.845098 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.826056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46261 (local address 127.5.78.3:37515)
0926 20:45:44.826829 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:44.826866 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:44.828493 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:44.829325 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:44.829338 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:44.830153 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:44.830829 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:44.830871 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.832017 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.832026 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:44.833918 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:44.833933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.834806 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.834813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:44.835030 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.835686 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:44.835716 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:44.838054 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:44.841089 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:44.841096 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:44.841109 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:44.841523 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:44.841890 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:44.841894 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:44.841896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:44.842075 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:44.842716 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:44.842722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:44.843049 (+ 327us) client_negotiation.cc:769] Sending connection context
0926 20:45:44.843297 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 20:45:44.843762 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":337,"threads_started":1}
I20260926 20:45:44.846580 5940 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46261
I20260926 20:45:44.846886 5940 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:44.847712 5940 heartbeater.cc:511] Master 127.5.78.62:46261 requested a full tablet report, sending...
I20260926 20:45:44.849076 5484 ts_manager.cc:194] Registered new tserver with Master: f9d9586376a441a380575c76234e33dc (127.5.78.3:46035)
I20260926 20:45:44.849882 5484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:37515
I20260926 20:45:44.858814 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:45:44.878037 5673 heartbeater.cc:503] Master 127.5.78.62:46261 was elected leader, sending a full tablet report...
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:45:44.907040 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.888467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.5.78.1:46693)
0926 20:45:44.889105 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:44.889113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:44.889196 (+ 83us) server_negotiation.cc:407] Connection header received
0926 20:45:44.889329 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:44.889336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:44.889561 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:44.889697 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:44.891294 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.892337 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.893574 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.894141 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.894250 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.898702 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:44.898715 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:44.898730 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:44.898767 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:44.903271 (+ 4504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.904401 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.904413 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.904417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.904639 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.905157 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.905164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.905167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.905714 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:44.906028 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:44.906367 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 20:45:44.906657 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":312,"threads_started":1}
W20260926 20:45:44.909605 5588 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58828
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:45:44 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:45:44.954736 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.939911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.5.78.1:46693)
0926 20:45:44.940168 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:44.940176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:44.940396 (+ 220us) server_negotiation.cc:407] Connection header received
0926 20:45:44.940619 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:44.940623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:44.940713 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:44.940825 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:44.942158 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.942957 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.944183 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.944595 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.944732 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.949573 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:44.949579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:44.949582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:44.949623 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:44.952308 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.952991 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.952996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.952998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.953130 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.953560 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.953564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.953566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.953826 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:44.954016 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:44.954261 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 20:45:44.954501 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 20:45:44.956331 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 5545
I20260926 20:45:44.969358 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:44.957253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.5.78.1:46693)
0926 20:45:44.957503 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:44.957510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:44.957772 (+ 262us) server_negotiation.cc:407] Connection header received
0926 20:45:44.957978 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:44.957990 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:44.958048 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:44.958147 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:44.959375 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.960080 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.961566 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:44.962049 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:44.962200 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:44.964655 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:44.964660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:44.964663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:44.964694 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:44.967257 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.967860 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.967864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.967866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.967996 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:44.968289 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:44.968292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:44.968294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:44.968531 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:44.968694 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:44.968933 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 20:45:44.969086 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:45:44.970018 5668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:45.052574 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 5545
I20260926 20:45:45.072852 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 5679
Sep 26 20:45:45 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:45:45.089838 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:45.073909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.5.78.2:42785)
0926 20:45:45.074590 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:45.074598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:45.074634 (+ 36us) server_negotiation.cc:407] Connection header received
0926 20:45:45.074696 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:45.074703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:45.074921 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:45.075058 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:45.076242 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:45.077275 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:45.078606 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:45.079157 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:45.079264 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:45.083660 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:45.083679 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:45.083726 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:45.083775 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:45.086960 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:45.087591 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:45.087599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:45.087601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:45.087715 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:45.088089 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:45.088093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:45.088095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:45.088476 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:45.088835 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:45.089201 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 20:45:45.089479 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":357,"threads_started":1}
I20260926 20:45:45.091228 5802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:45.174628 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 5679
I20260926 20:45:45.190338 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 5812
Sep 26 20:45:45 dist-test-slave-z7j8 krb5kdc[5439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:45:45.208511 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:45.191431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.5.78.3:46035)
0926 20:45:45.192119 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:45.192127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:45.192159 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:45:45.192227 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:45.192233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:45.192446 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:45.192584 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:45.194159 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:45.195309 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:45.197199 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:45.197861 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:45.198026 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:45.202593 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:45.202610 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:45.202626 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:45.202671 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:45.205504 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:45.206330 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:45.206339 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:45.206341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:45.206546 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:45.206941 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:45.206945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:45.206947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:45.207324 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:45.207594 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:45.207883 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 20:45:45.208094 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":371,"threads_started":1}
I20260926 20:45:45.209875 5935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:45.292088 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 5812
I20260926 20:45:45.307613 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 5452
I20260926 20:45:45.308993 5514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:45.411409 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 5452
2026-09-26T20:45:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3993 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:45 dist-test-slave-z7j8 krb5kdc[5961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:45 dist-test-slave-z7j8 krb5kdc[5961](info): set up 2 sockets
Sep 26 20:45:45 dist-test-slave-z7j8 krb5kdc[5961](info): commencing operation
krb5kdc: starting...
W20260926 20:45:46.494539 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:45:46 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:45:46.678391 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33463
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:47.006255 5974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:47.006706 5974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:47.006809 5974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:47.014237 5974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:47.014459 5974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:47.014562 5974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:47.014627 5974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:47.014669 5974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:47.024889 5974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:47.026199 5974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:47.027590 5974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:47.039554 5979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:47.039584 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:47.039584 5980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:47.040544 5974 server_base.cc:1034] running on GCE node
I20260926 20:45:47.041332 5974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:47.042907 5974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:47.044168 5974 hybrid_clock.cc:648] HybridClock initialized: now 1790455547044115 us; error 38 us; skew 500 ppm
I20260926 20:45:47.044626 5974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:47 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455547, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:47.048273 5974 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:45:47.049882 5974 webserver.cc:460] Webserver started at http://127.5.78.62:33717/ using document root <none> and password file <none>
I20260926 20:45:47.050698 5974 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:47.050806 5974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:47.051167 5974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:47.054214 5974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "436c063ce644426bb6c4d8242c6eb16f"
format_stamp: "Formatted at 2026-09-26 20:45:47 on dist-test-slave-z7j8"
server_key: "4d9d8281bc89f64702d444c00fbce593"
server_key_iv: "e18896d51058ee454c356df70d479be2"
server_key_version: "encryptionkey@0"
I20260926 20:45:47.055090 5974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "436c063ce644426bb6c4d8242c6eb16f"
format_stamp: "Formatted at 2026-09-26 20:45:47 on dist-test-slave-z7j8"
server_key: "4d9d8281bc89f64702d444c00fbce593"
server_key_iv: "e18896d51058ee454c356df70d479be2"
server_key_version: "encryptionkey@0"
I20260926 20:45:47.060585 5974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:47.065202 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:47.067199 5974 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:45:47.067445 5974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "436c063ce644426bb6c4d8242c6eb16f"
format_stamp: "Formatted at 2026-09-26 20:45:47 on dist-test-slave-z7j8"
server_key: "4d9d8281bc89f64702d444c00fbce593"
server_key_iv: "e18896d51058ee454c356df70d479be2"
server_key_version: "encryptionkey@0"
I20260926 20:45:47.067728 5974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:47.104836 5974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:47.109572 5974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:47.109825 5974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:47.129101 5974 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33463
I20260926 20:45:47.129137 6040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33463 every 8 connection(s)
I20260926 20:45:47.130581 5974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:47.134377 6041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:47.137171 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 5974
I20260926 20:45:47.137418 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:45:47.138218 5432 external_mini_cluster.cc:1428] Setting key 67b7a8ab96a3dc6d28fe6eea2596cfb9
I20260926 20:45:47.147672 6041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f: Bootstrap starting.
I20260926 20:45:47.151587 6041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:47.152947 6041 log.cc:824] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:45:47 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:47.156121 6041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f: No bootstrap required, opened a new log
I20260926 20:45:47.162714 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:47.140517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.5.78.62:33463)
0926 20:45:47.141332 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:47.141346 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:47.141387 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:45:47.143166 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:47.143190 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:47.143655 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:47.144047 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:47.145906 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:47.147370 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:47.149150 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:47.149891 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:47.150034 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:47.155191 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:47.155198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:47.155213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:47.155249 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:47.158461 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:47.159291 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:47.159299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:47.159305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:47.159476 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:47.159879 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:47.159883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:47.159888 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:47.160414 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:47.160626 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:47.161175 (+ 549us) server_negotiation.cc:299] Negotiation successful
0926 20:45:47.161572 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":386,"threads_started":1}
I20260926 20:45:47.165328 6041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436c063ce644426bb6c4d8242c6eb16f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33463 } }
I20260926 20:45:47.165786 6041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:47.165917 6041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436c063ce644426bb6c4d8242c6eb16f, State: Initialized, Role: FOLLOWER
I20260926 20:45:47.166561 6041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436c063ce644426bb6c4d8242c6eb16f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33463 } }
I20260926 20:45:47.166771 6041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:47.166925 6041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:47.167093 6041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:47.169243 6041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436c063ce644426bb6c4d8242c6eb16f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33463 } }
I20260926 20:45:47.169783 6041 leader_election.cc:302] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 436c063ce644426bb6c4d8242c6eb16f; no voters:
I20260926 20:45:47.170501 6041 leader_election.cc:288] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:47.170665 6046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:47.172043 6046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [term 1 LEADER]: Becoming Leader. State: Replica: 436c063ce644426bb6c4d8242c6eb16f, State: Running, Role: LEADER
I20260926 20:45:47.172701 6046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436c063ce644426bb6c4d8242c6eb16f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33463 } }
I20260926 20:45:47.172821 6041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:47.175763 6047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "436c063ce644426bb6c4d8242c6eb16f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436c063ce644426bb6c4d8242c6eb16f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33463 } } }
I20260926 20:45:47.176231 6047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:47.176793 6048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 436c063ce644426bb6c4d8242c6eb16f. Latest consensus state: current_term: 1 leader_uuid: "436c063ce644426bb6c4d8242c6eb16f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436c063ce644426bb6c4d8242c6eb16f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33463 } } }
I20260926 20:45:47.177063 6048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:47.178136 6055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:47.185037 6055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:47.193245 6055 catalog_manager.cc:1347] Generated new cluster ID: 3cd6a2ba6d6f4f5cbdfb72f41e5879d6
I20260926 20:45:47.193379 6055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:47.204532 6055 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:47.205992 6055 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:47.212893 6055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 436c063ce644426bb6c4d8242c6eb16f: Generated new TSK 0
I20260926 20:45:47.213773 6055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:45:47.258069 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33463
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:47.579281 6067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:47.579591 6067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:47.579751 6067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:47.587666 6067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:47.587839 6067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:47.588029 6067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:45:47.598196 6067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33463
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:47.599620 6067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:47.600992 6067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:47.613895 6072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:47.614092 6073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:47.614753 6067 server_base.cc:1034] running on GCE node
W20260926 20:45:47.614781 6075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:47.615383 6067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:47.616281 6067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:47.617508 6067 hybrid_clock.cc:648] HybridClock initialized: now 1790455547617463 us; error 40 us; skew 500 ppm
I20260926 20:45:47.617818 6067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:47 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455547, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:47.621078 6067 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:45:47.622583 6067 webserver.cc:460] Webserver started at http://127.5.78.1:33917/ using document root <none> and password file <none>
I20260926 20:45:47.623317 6067 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:47.623471 6067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:47.623809 6067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:47.626881 6067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "1b4745b05d524b0e858da7d737de53f1"
format_stamp: "Formatted at 2026-09-26 20:45:47 on dist-test-slave-z7j8"
server_key: "aec5e9f0d8cf8a1149f4bda3b5aa8e18"
server_key_iv: "71b57951988f1c670629a97a59e10469"
server_key_version: "encryptionkey@0"
I20260926 20:45:47.627704 6067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "1b4745b05d524b0e858da7d737de53f1"
format_stamp: "Formatted at 2026-09-26 20:45:47 on dist-test-slave-z7j8"
server_key: "aec5e9f0d8cf8a1149f4bda3b5aa8e18"
server_key_iv: "71b57951988f1c670629a97a59e10469"
server_key_version: "encryptionkey@0"
I20260926 20:45:47.633181 6067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:45:47.636982 6082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:47.638505 6067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 20:45:47.638695 6067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "1b4745b05d524b0e858da7d737de53f1"
format_stamp: "Formatted at 2026-09-26 20:45:47 on dist-test-slave-z7j8"
server_key: "aec5e9f0d8cf8a1149f4bda3b5aa8e18"
server_key_iv: "71b57951988f1c670629a97a59e10469"
server_key_version: "encryptionkey@0"
I20260926 20:45:47.638899 6067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:47.660637 6067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:47.664646 6067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:47.664942 6067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:47.665866 6067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:45:47.667207 6067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:47.667320 6067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:47.667474 6067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:47.667529 6067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:47.701040 6067 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:42273
I20260926 20:45:47.701128 6194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:42273 every 8 connection(s)
I20260926 20:45:47.702282 6067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:45:47.704900 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6067
I20260926 20:45:47.705173 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:45:47.705829 5432 external_mini_cluster.cc:1428] Setting key 84efc3daf2e5a03b63de97899f80a432
Sep 26 20:45:47 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455547, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:45:47.722692 6197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:47.704928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:54373 (local address 127.5.78.62:33463)
0926 20:45:47.705428 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:47.705436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:47.706818 (+ 1382us) server_negotiation.cc:407] Connection header received
0926 20:45:47.708497 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:47.708502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:47.708596 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:47.708737 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:47.710255 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:47.711050 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:47.712635 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:47.713048 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:47.713218 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:47.716990 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:47.716995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:47.716998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:47.717031 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:47.719596 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:47.720344 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:47.720351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:47.720354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:47.720484 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:47.720924 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:47.720932 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:47.720935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:47.721226 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:47.721382 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:47.722075 (+ 693us) server_negotiation.cc:299] Negotiation successful
0926 20:45:47.722290 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:45:47.723801 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:47.705537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33463 (local address 127.5.78.1:54373)
0926 20:45:47.706373 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:47.706446 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:47.708227 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:47.708989 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:47.708998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:47.709508 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:47.710078 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:47.710096 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:47.711206 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:47.711216 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:47.712466 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:47.712478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:47.713308 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:47.713315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:47.713554 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:47.714301 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:47.714330 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:47.716744 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:47.719747 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:47.719757 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:47.719777 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:47.720217 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:47.720609 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:47.720614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:47.720618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:47.720812 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:47.721358 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:47.721366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:47.721731 (+ 365us) client_negotiation.cc:769] Sending connection context
0926 20:45:47.722003 (+ 272us) client_negotiation.cc:240] Negotiation successful
0926 20:45:47.722587 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":330,"threads_started":1}
I20260926 20:45:47.725531 6195 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33463
I20260926 20:45:47.725904 6195 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:47.726894 6195 heartbeater.cc:511] Master 127.5.78.62:33463 requested a full tablet report, sending...
I20260926 20:45:47.729552 6006 ts_manager.cc:194] Registered new tserver with Master: 1b4745b05d524b0e858da7d737de53f1 (127.5.78.1:42273)
I20260926 20:45:47.731914 6006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:54373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:45:47.746999 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33463
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:48.083362 6201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:48.083699 6201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:48.083873 6201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:48.091481 6201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:48.091665 6201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:48.091856 6201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:45:48.102283 6201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33463
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:48.103847 6201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:48.105274 6201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:48.119915 6206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:48.120042 6207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:48.120066 6209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:48.121018 6201 server_base.cc:1034] running on GCE node
I20260926 20:45:48.121731 6201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:48.122843 6201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:48.124079 6201 hybrid_clock.cc:648] HybridClock initialized: now 1790455548124010 us; error 69 us; skew 500 ppm
I20260926 20:45:48.124428 6201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:48 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455548, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:48.127775 6201 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:45:48.129411 6201 webserver.cc:460] Webserver started at http://127.5.78.2:37665/ using document root <none> and password file <none>
I20260926 20:45:48.130133 6201 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:48.130261 6201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:48.130657 6201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:48.133745 6201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "8526d07cc33745ec8b28d0c8f7f9146f"
format_stamp: "Formatted at 2026-09-26 20:45:48 on dist-test-slave-z7j8"
server_key: "c01e6e50c65630b1f8038cf36c1ebf86"
server_key_iv: "974ce042bebf9c1dff3709e5e0a12d2a"
server_key_version: "encryptionkey@0"
I20260926 20:45:48.134670 6201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "8526d07cc33745ec8b28d0c8f7f9146f"
format_stamp: "Formatted at 2026-09-26 20:45:48 on dist-test-slave-z7j8"
server_key: "c01e6e50c65630b1f8038cf36c1ebf86"
server_key_iv: "974ce042bebf9c1dff3709e5e0a12d2a"
server_key_version: "encryptionkey@0"
I20260926 20:45:48.140587 6201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:48.144644 6216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:48.146274 6201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:45:48.146564 6201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "8526d07cc33745ec8b28d0c8f7f9146f"
format_stamp: "Formatted at 2026-09-26 20:45:48 on dist-test-slave-z7j8"
server_key: "c01e6e50c65630b1f8038cf36c1ebf86"
server_key_iv: "974ce042bebf9c1dff3709e5e0a12d2a"
server_key_version: "encryptionkey@0"
I20260926 20:45:48.146811 6201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:48.180686 6201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:48.185117 6201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:48.185423 6201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:48.186689 6201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:45:48.188130 6201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:48.188206 6201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:48.188339 6201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:48.188398 6201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:48.223318 6201 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:35443
I20260926 20:45:48.223369 6328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:35443 every 8 connection(s)
I20260926 20:45:48.224660 6201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:45:48.225504 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6201
I20260926 20:45:48.225720 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:45:48.226238 5432 external_mini_cluster.cc:1428] Setting key ea34447aec7c1a9bd229a6d9463495ac
Sep 26 20:45:48 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455548, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:45:48.246662 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.227704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:52023 (local address 127.5.78.62:33463)
0926 20:45:48.228170 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:48.228178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:48.229474 (+ 1296us) server_negotiation.cc:407] Connection header received
0926 20:45:48.231401 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:48.231406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:48.231497 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:48.231616 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:48.233159 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.234258 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.236088 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.236507 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.236643 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.240270 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:48.240275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:48.240279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:48.240311 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:48.243198 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:48.244008 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:48.244015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:48.244023 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:48.244150 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:48.244694 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:48.244697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:48.244700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:48.244976 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:48.245213 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:48.246028 (+ 815us) server_negotiation.cc:299] Negotiation successful
0926 20:45:48.246248 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":244,"threads_started":1}
I20260926 20:45:48.247716 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.228483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33463 (local address 127.5.78.2:52023)
0926 20:45:48.229315 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:48.229361 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:48.231120 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:48.231870 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:48.231880 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:48.232412 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:48.232985 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.233002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.234554 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.234567 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.235914 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.235926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.236761 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.236768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.237015 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.237741 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:48.237766 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:48.240029 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:48.243348 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:48.243357 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:48.243377 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:48.243873 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:48.244282 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:48.244287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:48.244291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:48.244545 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:48.245176 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:48.245184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:48.245652 (+ 468us) client_negotiation.cc:769] Sending connection context
0926 20:45:48.245936 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 20:45:48.246553 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":353,"threads_started":1}
I20260926 20:45:48.249415 6329 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33463
I20260926 20:45:48.249833 6329 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:48.250907 6329 heartbeater.cc:511] Master 127.5.78.62:33463 requested a full tablet report, sending...
I20260926 20:45:48.252907 6006 ts_manager.cc:194] Registered new tserver with Master: 8526d07cc33745ec8b28d0c8f7f9146f (127.5.78.2:35443)
I20260926 20:45:48.253765 6006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:52023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:45:48.274518 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33463
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:48.612289 6335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:48.612596 6335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:48.612699 6335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:48.620213 6335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:48.620352 6335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:48.620468 6335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:45:48.630561 6335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33463
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:48.631960 6335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:48.633260 6335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:48.646991 6340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:48.646883 6341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:48.646953 6343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:48.648483 6335 server_base.cc:1034] running on GCE node
I20260926 20:45:48.649155 6335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:48.650161 6335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:48.651353 6335 hybrid_clock.cc:648] HybridClock initialized: now 1790455548651285 us; error 66 us; skew 500 ppm
I20260926 20:45:48.651741 6335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:48 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455548, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:48.655369 6335 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:45:48.657160 6335 webserver.cc:460] Webserver started at http://127.5.78.3:37583/ using document root <none> and password file <none>
I20260926 20:45:48.657958 6335 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:48.658133 6335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:48.658583 6335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:48.661794 6335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "920a72f4fd634e868d6eed2f05b372b3"
format_stamp: "Formatted at 2026-09-26 20:45:48 on dist-test-slave-z7j8"
server_key: "a03ac6bcc2230a36432f3a957aff52fd"
server_key_iv: "1b10827630e16496b05e8086cb6d245f"
server_key_version: "encryptionkey@0"
I20260926 20:45:48.662767 6335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "920a72f4fd634e868d6eed2f05b372b3"
format_stamp: "Formatted at 2026-09-26 20:45:48 on dist-test-slave-z7j8"
server_key: "a03ac6bcc2230a36432f3a957aff52fd"
server_key_iv: "1b10827630e16496b05e8086cb6d245f"
server_key_version: "encryptionkey@0"
I20260926 20:45:48.668344 6335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:48.672259 6350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:48.673978 6335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:45:48.674170 6335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "920a72f4fd634e868d6eed2f05b372b3"
format_stamp: "Formatted at 2026-09-26 20:45:48 on dist-test-slave-z7j8"
server_key: "a03ac6bcc2230a36432f3a957aff52fd"
server_key_iv: "1b10827630e16496b05e8086cb6d245f"
server_key_version: "encryptionkey@0"
I20260926 20:45:48.674443 6335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:48.695365 6335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:48.700477 6335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:48.700834 6335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:48.701946 6335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:45:48.703524 6335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:48.703644 6335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:48.703814 6335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:48.703893 6335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:48.736055 6195 heartbeater.cc:503] Master 127.5.78.62:33463 was elected leader, sending a full tablet report...
I20260926 20:45:48.739562 6335 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:46307
I20260926 20:45:48.739599 6462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:46307 every 8 connection(s)
I20260926 20:45:48.741004 6335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:45:48.743417 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6335
I20260926 20:45:48.743614 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:45:48.744061 5432 external_mini_cluster.cc:1428] Setting key 8a10ec96e809201c690510bf50d578d7
Sep 26 20:45:48 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455548, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:48.761950 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.745300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:48423 (local address 127.5.78.62:33463)
0926 20:45:48.745479 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:48.745485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:48.745669 (+ 184us) server_negotiation.cc:407] Connection header received
0926 20:45:48.747821 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:48.747825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:48.747888 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:48.747972 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:48.749544 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.750350 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.752145 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.752553 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.752673 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.756203 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:48.756208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:48.756212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:48.756248 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:48.759066 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:48.759952 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:48.759956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:48.759958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:48.760079 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:48.760462 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:48.760465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:48.760467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:48.760738 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:48.760844 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:48.761536 (+ 692us) server_negotiation.cc:299] Negotiation successful
0926 20:45:48.761728 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:45:48.763021 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.744550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33463 (local address 127.5.78.3:48423)
0926 20:45:48.745486 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:48.745544 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:48.747543 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:48.748249 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:48.748258 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:48.748818 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:48.749384 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.749402 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.750550 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.750553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.751910 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.751922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.752770 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.752776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.752976 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.753635 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:48.753662 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:48.755943 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:48.759217 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:48.759228 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:48.759248 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:48.759827 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:48.760176 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:48.760180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:48.760182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:48.760369 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:48.760859 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:48.760866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:48.761196 (+ 330us) client_negotiation.cc:769] Sending connection context
0926 20:45:48.761434 (+ 238us) client_negotiation.cc:240] Negotiation successful
0926 20:45:48.761840 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":831,"threads_started":1}
I20260926 20:45:48.764657 6463 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33463
I20260926 20:45:48.764981 6463 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:48.765686 6463 heartbeater.cc:511] Master 127.5.78.62:33463 requested a full tablet report, sending...
I20260926 20:45:48.767409 6006 ts_manager.cc:194] Registered new tserver with Master: 920a72f4fd634e868d6eed2f05b372b3 (127.5.78.3:46307)
I20260926 20:45:48.768227 6006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:48423
I20260926 20:45:48.772436 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:45:48.792459 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.779315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.5.78.62:33463)
0926 20:45:48.779496 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:48.779501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:48.780003 (+ 502us) server_negotiation.cc:407] Connection header received
0926 20:45:48.780193 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:48.780197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:48.780266 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:48.780366 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:48.781806 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.782663 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.784048 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.784498 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.784613 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.787112 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:48.787118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:48.787121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:48.787159 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:48.789820 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:48.790533 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:48.790537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:48.790539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:48.790678 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:48.791064 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:48.791068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:48.791069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:48.791387 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:48.791589 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:48.792014 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 20:45:48.792194 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:45:48.799084 6006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:45:48.805907 6006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:48.830530 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.818430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42273 (local address 127.0.0.1:46242)
0926 20:45:48.819605 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:48.819694 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:48.819893 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:48.820994 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:48.821002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:48.821031 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:48.821491 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.821508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.824755 (+ 3247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.824763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.827512 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.827524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.829286 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.829292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.829609 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.829691 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 20:45:48.829837 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 20:45:48.830036 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":64,"thread_start_us":1961,"threads_started":1}
I20260926 20:45:48.830614 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.818712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.5.78.1:42273)
0926 20:45:48.820293 (+ 1581us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:48.820304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:48.820337 (+ 33us) server_negotiation.cc:407] Connection header received
0926 20:45:48.820453 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:48.820463 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:48.820714 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:48.822580 (+ 1866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:48.822914 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.824566 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.827695 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.828693 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.828861 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.829138 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:48.830043 (+ 905us) server_negotiation.cc:299] Negotiation successful
0926 20:45:48.830237 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405,"thread_start_us":1386,"threads_started":1}
I20260926 20:45:48.831523 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.818441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:46307 (local address 127.0.0.1:53350)
0926 20:45:48.820682 (+ 2241us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:48.820704 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:48.820822 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:48.824399 (+ 3577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:48.824404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:48.824429 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:48.824864 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.824877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.826719 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.826727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.828835 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.828850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.830694 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.830700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.830871 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.830903 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:45:48.830967 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 20:45:48.831103 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2114,"mutex_wait_us":52,"thread_start_us":645,"threads_started":1}
I20260926 20:45:48.832360 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.818852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.5.78.3:46307)
0926 20:45:48.823780 (+ 4928us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:48.823791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:48.823829 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:45:48.823930 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:48.823938 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:48.824192 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:48.824411 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:48.825093 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.826589 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.829023 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.830486 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.831504 (+ 1018us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.831739 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:48.831812 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 20:45:48.831976 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4725,"thread_start_us":4556,"threads_started":1}
I20260926 20:45:48.832623 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.820421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:35443 (local address 127.0.0.1:57508)
0926 20:45:48.822179 (+ 1758us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:48.822200 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:48.822361 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:48.824763 (+ 2402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:48.824767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:48.824793 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:48.825404 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.825417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.827353 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.827359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.829576 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:48.829589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.831859 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.831865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:48.832045 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.832066 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 20:45:48.832173 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:45:48.832294 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1602,"mutex_wait_us":1,"thread_start_us":1420,"threads_started":1}
I20260926 20:45:48.832839 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:48.821101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.5.78.2:35443)
0926 20:45:48.823991 (+ 2890us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:48.824000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:48.824025 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:45:48.824118 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:48.824124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:48.824344 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:48.824530 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:48.825620 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.827197 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.829771 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:48.831355 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:48.831487 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:48.831719 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:48.832228 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 20:45:48.832382 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2753,"thread_start_us":2618,"threads_started":1}
I20260926 20:45:48.835227 6130 tablet_service.cc:1467] Processing CreateTablet for tablet 317f45aff2cf4a9b8f6501fd76e80d96 (DEFAULT_TABLE table=test-table [id=e8bc52e3198e4ab3b13775f1f11fc887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:48.836581 6130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317f45aff2cf4a9b8f6501fd76e80d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:48.836050 6264 tablet_service.cc:1467] Processing CreateTablet for tablet 317f45aff2cf4a9b8f6501fd76e80d96 (DEFAULT_TABLE table=test-table [id=e8bc52e3198e4ab3b13775f1f11fc887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:48.836050 6398 tablet_service.cc:1467] Processing CreateTablet for tablet 317f45aff2cf4a9b8f6501fd76e80d96 (DEFAULT_TABLE table=test-table [id=e8bc52e3198e4ab3b13775f1f11fc887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:48.837455 6264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317f45aff2cf4a9b8f6501fd76e80d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:48.837450 6398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317f45aff2cf4a9b8f6501fd76e80d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:48.849788 6481 tablet_bootstrap.cc:492] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: Bootstrap starting.
I20260926 20:45:48.852206 6483 tablet_bootstrap.cc:492] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: Bootstrap starting.
I20260926 20:45:48.855650 6481 tablet_bootstrap.cc:654] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:48.856318 6483 tablet_bootstrap.cc:654] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:48.856657 6482 tablet_bootstrap.cc:492] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: Bootstrap starting.
I20260926 20:45:48.857170 6481 log.cc:824] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:48.857949 6483 log.cc:824] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:48.860111 6482 tablet_bootstrap.cc:654] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:48.860471 6481 tablet_bootstrap.cc:492] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: No bootstrap required, opened a new log
I20260926 20:45:48.860785 6481 ts_tablet_manager.cc:1397] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260926 20:45:48.861446 6482 log.cc:824] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:48.862969 6483 tablet_bootstrap.cc:492] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: No bootstrap required, opened a new log
I20260926 20:45:48.863276 6483 ts_tablet_manager.cc:1397] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260926 20:45:48.864384 6482 tablet_bootstrap.cc:492] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: No bootstrap required, opened a new log
I20260926 20:45:48.864651 6482 ts_tablet_manager.cc:1397] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 20:45:48.871979 6482 raft_consensus.cc:348] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:48.872577 6482 raft_consensus.cc:374] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:48.872725 6482 raft_consensus.cc:729] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8526d07cc33745ec8b28d0c8f7f9146f, State: Initialized, Role: FOLLOWER
I20260926 20:45:48.873458 6482 consensus_queue.cc:260] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:48.873250 6481 raft_consensus.cc:348] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:48.873837 6481 raft_consensus.cc:374] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:48.874042 6481 raft_consensus.cc:729] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4745b05d524b0e858da7d737de53f1, State: Initialized, Role: FOLLOWER
I20260926 20:45:48.875097 6329 heartbeater.cc:503] Master 127.5.78.62:33463 was elected leader, sending a full tablet report...
I20260926 20:45:48.875494 6481 consensus_queue.cc:260] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:48.875451 6483 raft_consensus.cc:348] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:48.875837 6482 ts_tablet_manager.cc:1428] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: Time spent starting tablet: real 0.011s user 0.001s sys 0.011s
I20260926 20:45:48.876040 6483 raft_consensus.cc:374] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:48.876156 6483 raft_consensus.cc:729] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 920a72f4fd634e868d6eed2f05b372b3, State: Initialized, Role: FOLLOWER
I20260926 20:45:48.876926 6483 consensus_queue.cc:260] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:48.877630 6481 ts_tablet_manager.cc:1428] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260926 20:45:48.878679 6463 heartbeater.cc:503] Master 127.5.78.62:33463 was elected leader, sending a full tablet report...
I20260926 20:45:48.879911 6483 ts_tablet_manager.cc:1428] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
W20260926 20:45:48.954704 6196 tablet.cc:2369] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:45:48.976995 6330 tablet.cc:2369] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:45:48.993196 6464 tablet.cc:2369] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:49.165881 6489 raft_consensus.cc:482] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:49.166200 6489 raft_consensus.cc:504] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:49.168009 6489 leader_election.cc:288] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b4745b05d524b0e858da7d737de53f1 (127.5.78.1:42273), 8526d07cc33745ec8b28d0c8f7f9146f (127.5.78.2:35443)
I20260926 20:45:49.175092 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.168553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:47197 (local address 127.5.78.1:42273)
0926 20:45:49.168754 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.168760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.168799 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:45:49.169066 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.169072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.169158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.169270 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:49.169816 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.170793 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.172688 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.174066 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.174230 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.174314 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.174693 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.174802 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:45:49.175097 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.168363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42273 (local address 127.5.78.3:47197)
0926 20:45:49.168698 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:49.168721 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:49.168891 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:49.169303 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:49.169307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:49.169330 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:49.169634 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:49.169645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.170917 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.170923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:49.172480 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:49.172493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.174244 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.174251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:49.174494 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.174531 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:45:49.174639 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 20:45:49.174786 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"spinlock_wait_cycles":44800}
I20260926 20:45:49.176116 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.168551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:46215 (local address 127.5.78.2:35443)
0926 20:45:49.168754 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.168760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.169430 (+ 670us) server_negotiation.cc:407] Connection header received
0926 20:45:49.169595 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.169602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.169676 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.169796 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:49.170389 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.171384 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.173871 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.175190 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.175330 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.175432 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.175829 (+ 397us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.175938 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:45:49.176328 6490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.168418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:35443 (local address 127.5.78.3:46215)
0926 20:45:49.169303 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:49.169323 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:49.169464 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:49.169828 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:49.169833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:49.169870 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:49.170195 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:49.170206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.171569 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.171578 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:49.173588 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:49.173603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.175541 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.175547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:49.175675 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.175698 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:45:49.175777 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 20:45:49.175928 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":490,"threads_started":1}
I20260926 20:45:49.176373 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f45aff2cf4a9b8f6501fd76e80d96" candidate_uuid: "920a72f4fd634e868d6eed2f05b372b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4745b05d524b0e858da7d737de53f1" is_pre_election: true
I20260926 20:45:49.177043 6150 raft_consensus.cc:2454] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 920a72f4fd634e868d6eed2f05b372b3 in term 0.
I20260926 20:45:49.177543 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f45aff2cf4a9b8f6501fd76e80d96" candidate_uuid: "920a72f4fd634e868d6eed2f05b372b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" is_pre_election: true
I20260926 20:45:49.177927 6352 leader_election.cc:302] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4745b05d524b0e858da7d737de53f1, 920a72f4fd634e868d6eed2f05b372b3; no voters:
I20260926 20:45:49.178013 6284 raft_consensus.cc:2454] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 920a72f4fd634e868d6eed2f05b372b3 in term 0.
I20260926 20:45:49.178372 6489 raft_consensus.cc:2781] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:49.178609 6489 raft_consensus.cc:482] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:49.178764 6489 raft_consensus.cc:3037] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:49.181103 6489 raft_consensus.cc:504] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:49.181821 6489 leader_election.cc:288] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [CANDIDATE]: Term 1 election: Requested vote from peers 1b4745b05d524b0e858da7d737de53f1 (127.5.78.1:42273), 8526d07cc33745ec8b28d0c8f7f9146f (127.5.78.2:35443)
I20260926 20:45:49.182317 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f45aff2cf4a9b8f6501fd76e80d96" candidate_uuid: "920a72f4fd634e868d6eed2f05b372b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4745b05d524b0e858da7d737de53f1"
I20260926 20:45:49.182371 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f45aff2cf4a9b8f6501fd76e80d96" candidate_uuid: "920a72f4fd634e868d6eed2f05b372b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8526d07cc33745ec8b28d0c8f7f9146f"
I20260926 20:45:49.182590 6150 raft_consensus.cc:3037] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:49.182590 6284 raft_consensus.cc:3037] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:49.185865 6284 raft_consensus.cc:2454] T 317f45aff2cf4a9b8f6501fd76e80d96 P 8526d07cc33745ec8b28d0c8f7f9146f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 920a72f4fd634e868d6eed2f05b372b3 in term 1.
I20260926 20:45:49.185866 6150 raft_consensus.cc:2454] T 317f45aff2cf4a9b8f6501fd76e80d96 P 1b4745b05d524b0e858da7d737de53f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 920a72f4fd634e868d6eed2f05b372b3 in term 1.
I20260926 20:45:49.186446 6353 leader_election.cc:302] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8526d07cc33745ec8b28d0c8f7f9146f, 920a72f4fd634e868d6eed2f05b372b3; no voters:
I20260926 20:45:49.186820 6489 raft_consensus.cc:2781] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:49.187219 6489 raft_consensus.cc:686] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [term 1 LEADER]: Becoming Leader. State: Replica: 920a72f4fd634e868d6eed2f05b372b3, State: Running, Role: LEADER
I20260926 20:45:49.187712 6489 consensus_queue.cc:237] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } }
I20260926 20:45:49.192384 6005 catalog_manager.cc:5495] T 317f45aff2cf4a9b8f6501fd76e80d96 P 920a72f4fd634e868d6eed2f05b372b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 920a72f4fd634e868d6eed2f05b372b3 (127.5.78.3). New cstate: current_term: 1 leader_uuid: "920a72f4fd634e868d6eed2f05b372b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4745b05d524b0e858da7d737de53f1" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "920a72f4fd634e868d6eed2f05b372b3" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 46307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8526d07cc33745ec8b28d0c8f7f9146f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 35443 } health_report { overall_health: UNKNOWN } } }
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:45:49.296000 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.280559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.5.78.1:42273)
0926 20:45:49.280743 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.280748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.281269 (+ 521us) server_negotiation.cc:407] Connection header received
0926 20:45:49.281396 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.281400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.281484 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.281577 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:49.283067 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.283826 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.285265 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.285769 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.285895 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.290187 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:49.290205 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:49.290222 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:49.290266 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:49.293445 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.294117 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.294125 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.294128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.294267 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.294689 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.294693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.294695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.295079 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:49.295321 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.295547 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.295709 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:45:49.341560 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.327672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46254 (local address 127.5.78.1:42273)
0926 20:45:49.327946 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.327952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.328210 (+ 258us) server_negotiation.cc:407] Connection header received
0926 20:45:49.328325 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.328329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.328394 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.328491 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:49.329700 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.330487 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.331906 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.332423 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.332521 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.336437 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:49.336443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:49.336446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:49.336482 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:49.339056 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.339731 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.339736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.339738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.339858 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.340240 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.340243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.340245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.340521 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:49.340694 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.341073 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.341239 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:45:49.387121 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.373200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.5.78.1:42273)
0926 20:45:49.373460 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.373468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.373744 (+ 276us) server_negotiation.cc:407] Connection header received
0926 20:45:49.374015 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.374021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.374091 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.374202 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:49.375742 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.376483 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.377707 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.378190 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.378328 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.382354 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:49.382360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:49.382363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:49.382396 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:49.384672 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.385304 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.385308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.385310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.385428 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.385730 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.385733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.385735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.386003 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:49.386173 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.386667 (+ 494us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.386860 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 20:45:49.388783 6110 tablet_service.cc:2908] Rejecting scan request for tablet 317f45aff2cf4a9b8f6501fd76e80d96: Uninitialized: clean time has not yet been initialized
I20260926 20:45:49.392288 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6067
I20260926 20:45:49.405452 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.393204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.5.78.1:42273)
0926 20:45:49.393407 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.393420 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.393732 (+ 312us) server_negotiation.cc:407] Connection header received
0926 20:45:49.393874 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.393879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.393942 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.394034 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:49.395251 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.395991 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.397162 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.397581 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.397698 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.400298 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:49.400303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:49.400307 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:49.400341 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:49.403040 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.403732 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.403737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.403740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.403859 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.404193 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.404197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.404199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.404464 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:49.404669 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.405029 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.405200 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:45:49.406175 6190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:49.493877 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6067
I20260926 20:45:49.510217 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6201
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:45:49.525844 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.511079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.5.78.2:35443)
0926 20:45:49.511296 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.511301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.511320 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:45:49.511411 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.511414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.511477 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.511570 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:49.512828 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.513583 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.515144 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.515548 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.515664 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.519901 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:49.519923 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:49.519941 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:49.519986 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:49.522945 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.523764 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.523773 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.523775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.523910 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.524280 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.524283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.524285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.524669 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:49.524893 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.525273 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.525451 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 20:45:49.526600 6324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:49.617079 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6201
I20260926 20:45:49.633539 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6335
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:45:49.649320 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.634354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53354 (local address 127.5.78.3:46307)
0926 20:45:49.634907 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:49.634912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:49.634930 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:45:49.634986 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:49.634990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:49.635058 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:49.635163 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:49.636381 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.637113 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.638330 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:49.638958 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:49.639133 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:49.643422 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:49.643438 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:49.643453 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:49.643489 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:49.646304 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.647111 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.647125 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.647127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.647288 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:49.647770 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:49.647774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:49.647777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:49.648212 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:49.648463 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:49.648711 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 20:45:49.648890 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":331,"threads_started":1}
I20260926 20:45:49.650136 6458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:49.739102 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6335
I20260926 20:45:49.739461 6490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:49.650195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42273 (local address 127.5.78.3:35723)
0926 20:45:49.739075 (+ 88880us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:49.739265 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 20:45:49.757311 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 5974
I20260926 20:45:49.758550 6036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:49.864480 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 5974
2026-09-26T20:45:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4454 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[6511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[6511](info): set up 2 sockets
Sep 26 20:45:49 dist-test-slave-z7j8 krb5kdc[6511](info): commencing operation
krb5kdc: starting...
W20260926 20:45:50.962502 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:45:51 dist-test-slave-z7j8 krb5kdc[6511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:45:51.158955 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33839
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:51.488915 6524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:51.489229 6524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:51.489380 6524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:51.496958 6524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:51.497085 6524 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 20:45:51.497162 6524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:51.497249 6524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:51.497327 6524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:51.497440 6524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:51.507833 6524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33839
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:51.509213 6524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:51.510674 6524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:51.522569 6530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:51.522590 6529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:51.522617 6532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:51.523290 6524 server_base.cc:1034] running on GCE node
I20260926 20:45:51.524183 6524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:51.525674 6524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:51.526959 6524 hybrid_clock.cc:648] HybridClock initialized: now 1790455551526887 us; error 40 us; skew 500 ppm
I20260926 20:45:51.527374 6524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:51 dist-test-slave-z7j8 krb5kdc[6511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455551, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:51.530917 6524 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:45:51.532608 6524 webserver.cc:460] Webserver started at http://127.5.78.62:34777/ using document root <none> and password file <none>
I20260926 20:45:51.533375 6524 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:51.533524 6524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:51.533879 6524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:51.537307 6524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "f991b543dd4545b1a2843b9bfdc852e0"
format_stamp: "Formatted at 2026-09-26 20:45:51 on dist-test-slave-z7j8"
server_key: "c2218087e17ab2e3cd18e3d576730fcb"
server_key_iv: "dd31b13050225230921416c23deb97e9"
server_key_version: "encryptionkey@0"
I20260926 20:45:51.538262 6524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "f991b543dd4545b1a2843b9bfdc852e0"
format_stamp: "Formatted at 2026-09-26 20:45:51 on dist-test-slave-z7j8"
server_key: "c2218087e17ab2e3cd18e3d576730fcb"
server_key_iv: "dd31b13050225230921416c23deb97e9"
server_key_version: "encryptionkey@0"
I20260926 20:45:51.544160 6524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:51.548084 6539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:51.549508 6524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:45:51.549722 6524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "f991b543dd4545b1a2843b9bfdc852e0"
format_stamp: "Formatted at 2026-09-26 20:45:51 on dist-test-slave-z7j8"
server_key: "c2218087e17ab2e3cd18e3d576730fcb"
server_key_iv: "dd31b13050225230921416c23deb97e9"
server_key_version: "encryptionkey@0"
I20260926 20:45:51.549957 6524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:51.593854 6524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:51.598522 6524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:51.598821 6524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:51.621923 6524 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33839
I20260926 20:45:51.621941 6600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33839 every 8 connection(s)
I20260926 20:45:51.623416 6524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:51.627990 6601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:51.628367 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6524
I20260926 20:45:51.628651 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:45:51.629602 5432 external_mini_cluster.cc:1428] Setting key e80baaadcb5098c9e732c9ff5c5925e1
I20260926 20:45:51.644019 6601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: Bootstrap starting.
Sep 26 20:45:51 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:51.648087 6601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:51.649515 6601 log.cc:824] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:51.653102 6601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: No bootstrap required, opened a new log
I20260926 20:45:51.657377 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:51.632113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.5.78.62:33839)
0926 20:45:51.633076 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:51.633089 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:51.633139 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:45:51.635603 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:51.635635 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:51.636135 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:51.636545 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:51.638400 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:51.639949 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:51.641868 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:51.642730 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:51.642886 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:51.648123 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:51.648135 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:51.648159 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:51.648220 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:51.652238 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:51.652997 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:51.653006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:51.653015 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:51.653278 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:51.653794 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:51.653799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:51.653802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:51.654595 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:51.654893 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:51.655359 (+ 466us) server_negotiation.cc:299] Negotiation successful
0926 20:45:51.655874 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":457,"threads_started":1}
I20260926 20:45:51.662937 6601 raft_consensus.cc:348] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f991b543dd4545b1a2843b9bfdc852e0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33839 } }
I20260926 20:45:51.663306 6601 raft_consensus.cc:374] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:51.663463 6601 raft_consensus.cc:729] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f991b543dd4545b1a2843b9bfdc852e0, State: Initialized, Role: FOLLOWER
I20260926 20:45:51.664179 6601 consensus_queue.cc:260] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f991b543dd4545b1a2843b9bfdc852e0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33839 } }
I20260926 20:45:51.664453 6601 raft_consensus.cc:388] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:51.664600 6601 raft_consensus.cc:482] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:51.664777 6601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:51.667088 6601 raft_consensus.cc:504] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f991b543dd4545b1a2843b9bfdc852e0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33839 } }
I20260926 20:45:51.667729 6601 leader_election.cc:302] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f991b543dd4545b1a2843b9bfdc852e0; no voters:
I20260926 20:45:51.668262 6601 leader_election.cc:288] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:51.668426 6606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:51.669149 6606 raft_consensus.cc:686] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [term 1 LEADER]: Becoming Leader. State: Replica: f991b543dd4545b1a2843b9bfdc852e0, State: Running, Role: LEADER
I20260926 20:45:51.669734 6606 consensus_queue.cc:237] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f991b543dd4545b1a2843b9bfdc852e0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33839 } }
I20260926 20:45:51.670526 6601 sys_catalog.cc:564] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:51.673760 6607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f991b543dd4545b1a2843b9bfdc852e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f991b543dd4545b1a2843b9bfdc852e0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33839 } } }
I20260926 20:45:51.673770 6608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f991b543dd4545b1a2843b9bfdc852e0. Latest consensus state: current_term: 1 leader_uuid: "f991b543dd4545b1a2843b9bfdc852e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f991b543dd4545b1a2843b9bfdc852e0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33839 } } }
I20260926 20:45:51.674235 6608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:51.674237 6607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:51.678967 6615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:51.684478 6615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:51.694216 6615 catalog_manager.cc:1347] Generated new cluster ID: 35529d09d1f64f0e9b8610aca9dd9ec1
I20260926 20:45:51.694351 6615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:51.718147 6615 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:51.719796 6615 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:51.729671 6615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: Generated new TSK 0
I20260926 20:45:51.730680 6615 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:45:51.775991 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33839
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:52.109148 6627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:52.109429 6627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:52.109578 6627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:52.117009 6627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:52.117144 6627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:52.117285 6627 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:45:52.117429 6627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:45:52.127519 6627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:33839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:52.128907 6627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:52.130354 6627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:52.144428 6633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:52.144518 6632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:52.144518 6635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:52.145164 6627 server_base.cc:1034] running on GCE node
I20260926 20:45:52.145851 6627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:52.147027 6627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:52.148304 6627 hybrid_clock.cc:648] HybridClock initialized: now 1790455552148250 us; error 39 us; skew 500 ppm
I20260926 20:45:52.148677 6627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:52 dist-test-slave-z7j8 krb5kdc[6511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455552, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:52.152266 6627 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:45:52.154112 6627 webserver.cc:460] Webserver started at http://127.5.78.1:34939/ using document root <none> and password file <none>
I20260926 20:45:52.154975 6627 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:52.155114 6627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:52.155686 6627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:52.159113 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "a977f19a97be4494923ffd45f88b5bb4"
format_stamp: "Formatted at 2026-09-26 20:45:52 on dist-test-slave-z7j8"
server_key: "7bda8a269e174321163ad1dd7aa10bcc"
server_key_iv: "a7a85ed5fcebfa4e2eb7f5c5d61305cd"
server_key_version: "encryptionkey@0"
I20260926 20:45:52.159932 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "a977f19a97be4494923ffd45f88b5bb4"
format_stamp: "Formatted at 2026-09-26 20:45:52 on dist-test-slave-z7j8"
server_key: "7bda8a269e174321163ad1dd7aa10bcc"
server_key_iv: "a7a85ed5fcebfa4e2eb7f5c5d61305cd"
server_key_version: "encryptionkey@0"
I20260926 20:45:52.165616 6627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:45:52.169667 6642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:52.171259 6627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 20:45:52.171459 6627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "a977f19a97be4494923ffd45f88b5bb4"
format_stamp: "Formatted at 2026-09-26 20:45:52 on dist-test-slave-z7j8"
server_key: "7bda8a269e174321163ad1dd7aa10bcc"
server_key_iv: "a7a85ed5fcebfa4e2eb7f5c5d61305cd"
server_key_version: "encryptionkey@0"
I20260926 20:45:52.171756 6627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:52.187325 6627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:52.192225 6627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:52.192536 6627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:52.195713 6627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:52.195837 6627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:52.196013 6627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:52.196107 6627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:45:52 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455552, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:52.221279 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.196902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:32823 (local address 127.5.78.62:33839)
0926 20:45:52.197512 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:52.197518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:52.199364 (+ 1846us) server_negotiation.cc:407] Connection header received
0926 20:45:52.201184 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:52.201190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:52.201287 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:52.201427 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:52.203823 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.205354 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.207884 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.208728 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.208888 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.214247 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:52.214256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:52.214262 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:52.214312 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:52.217395 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.218659 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.218665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.218669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.218823 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.219480 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.219485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.219493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.219779 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:52.219939 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:52.220698 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 20:45:52.220920 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":293,"threads_started":1}
I20260926 20:45:52.222667 6652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.197650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.5.78.1:32823)
0926 20:45:52.198702 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:52.198741 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:52.200814 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:52.201696 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:52.201706 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:52.202589 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:52.203495 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.203519 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.205601 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.205608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.207619 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.207644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.209317 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.209327 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.209716 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.210537 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:52.210562 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:52.213862 (+ 3300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:52.217573 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.217589 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.217616 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.218479 (+ 863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.218970 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.218975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.218979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.219312 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.219917 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:52.219925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:52.220329 (+ 404us) client_negotiation.cc:769] Sending connection context
0926 20:45:52.220600 (+ 271us) client_negotiation.cc:240] Negotiation successful
0926 20:45:52.221183 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":453,"threads_started":1}
I20260926 20:45:52.245671 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.228542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:54097 (local address 127.5.78.62:33839)
0926 20:45:52.228759 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:52.228767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:52.229853 (+ 1086us) server_negotiation.cc:407] Connection header received
0926 20:45:52.230064 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:52.230068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:52.230129 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:52.230219 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:52.232119 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.232824 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.235369 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.235769 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.235867 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.238806 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:52.238815 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:52.238821 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:52.238883 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:52.242366 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.243364 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.243369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.243373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.243552 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.244121 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.244129 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.244134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.244517 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:52.244667 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:52.245106 (+ 439us) server_negotiation.cc:299] Negotiation successful
0926 20:45:52.245328 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:45:52.245764 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.228542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.5.78.1:54097)
0926 20:45:52.229720 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:52.229744 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:52.229930 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:52.230289 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:52.230297 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:52.231232 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:52.231874 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.231888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.232974 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.232982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.235180 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.235213 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.235989 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.235999 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.236297 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.237067 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:52.237094 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:52.238583 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:52.242584 (+ 4001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.242593 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.242598 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.243222 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.243707 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.243715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.243722 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.243974 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.244635 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:52.244643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:52.244840 (+ 197us) client_negotiation.cc:769] Sending connection context
0926 20:45:52.244975 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 20:45:52.245315 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"thread_start_us":560,"threads_started":1}
I20260926 20:45:52.256915 6627 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:44399
I20260926 20:45:52.256935 6762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:44399 every 8 connection(s)
I20260926 20:45:52.258113 6627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:45:52.260070 6763 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33839
I20260926 20:45:52.260315 6763 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:52.260924 6763 heartbeater.cc:511] Master 127.5.78.62:33839 requested a full tablet report, sending...
I20260926 20:45:52.262753 6556 ts_manager.cc:194] Registered new tserver with Master: a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1:44399)
I20260926 20:45:52.264178 6556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:32823
I20260926 20:45:52.265779 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6627
I20260926 20:45:52.265980 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:45:52.266624 5432 external_mini_cluster.cc:1428] Setting key 51f0a00cb43d690b3c10fbf7508b21e6
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:45:52.307405 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33839
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:52.644968 6767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:52.645345 6767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:52.645490 6767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:52.653100 6767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:52.653234 6767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:52.653326 6767 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:45:52.653450 6767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:45:52.663551 6767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:33839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:52.664884 6767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:52.666211 6767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:52.680219 6773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:52.680254 6772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:52.680222 6775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:52.681109 6767 server_base.cc:1034] running on GCE node
I20260926 20:45:52.681748 6767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:52.682814 6767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:52.684085 6767 hybrid_clock.cc:648] HybridClock initialized: now 1790455552684024 us; error 47 us; skew 500 ppm
I20260926 20:45:52.684429 6767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:52 dist-test-slave-z7j8 krb5kdc[6511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455552, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:52.687851 6767 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:45:52.689445 6767 webserver.cc:460] Webserver started at http://127.5.78.2:35057/ using document root <none> and password file <none>
I20260926 20:45:52.690171 6767 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:52.690306 6767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:52.690699 6767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:52.693665 6767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "f91d09ae3f954806995fb1e60a2cb4cd"
format_stamp: "Formatted at 2026-09-26 20:45:52 on dist-test-slave-z7j8"
server_key: "b0a12bff097a5fe65cbd99cb12a93d68"
server_key_iv: "13c4096c6fd65d19e8fc3160eda3b3c1"
server_key_version: "encryptionkey@0"
I20260926 20:45:52.694566 6767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "f91d09ae3f954806995fb1e60a2cb4cd"
format_stamp: "Formatted at 2026-09-26 20:45:52 on dist-test-slave-z7j8"
server_key: "b0a12bff097a5fe65cbd99cb12a93d68"
server_key_iv: "13c4096c6fd65d19e8fc3160eda3b3c1"
server_key_version: "encryptionkey@0"
I20260926 20:45:52.699906 6767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 20:45:52.703503 6782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:52.704870 6767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:45:52.705045 6767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "f91d09ae3f954806995fb1e60a2cb4cd"
format_stamp: "Formatted at 2026-09-26 20:45:52 on dist-test-slave-z7j8"
server_key: "b0a12bff097a5fe65cbd99cb12a93d68"
server_key_iv: "13c4096c6fd65d19e8fc3160eda3b3c1"
server_key_version: "encryptionkey@0"
I20260926 20:45:52.705268 6767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:52.722337 6767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:52.727175 6767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:52.727465 6767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:52.730382 6767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:52.730516 6767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 20:45:52.730651 6767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:52.730731 6767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:45:52 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455552, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:52.754376 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.731216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:60795 (local address 127.5.78.62:33839)
0926 20:45:52.731409 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:52.731415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:52.732826 (+ 1411us) server_negotiation.cc:407] Connection header received
0926 20:45:52.735023 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:52.735033 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:52.735139 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:52.735274 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:52.737824 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.738990 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.740714 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.741125 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.741272 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.746626 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:52.746632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:52.746636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:52.746671 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:52.750494 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.751509 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.751516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.751520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.751667 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.752173 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.752181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.752185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.752513 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:52.752681 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:52.753604 (+ 923us) server_negotiation.cc:299] Negotiation successful
0926 20:45:52.753895 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:45:52.756392 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.731735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.5.78.2:60795)
0926 20:45:52.732625 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:52.732691 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:52.734690 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:52.735563 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:52.735573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:52.736364 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:52.737594 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.737635 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.739152 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.739165 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.740541 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.740555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.741381 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.741390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.741635 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.742397 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:52.742475 (+ 78us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:52.746294 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:52.750700 (+ 4406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.750714 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.750741 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.751354 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.751805 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.751811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.751817 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.752041 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.752669 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:52.752679 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:52.753065 (+ 386us) client_negotiation.cc:769] Sending connection context
0926 20:45:52.753526 (+ 461us) client_negotiation.cc:240] Negotiation successful
0926 20:45:52.754238 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":376,"threads_started":1}
I20260926 20:45:52.778973 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.762977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.5.78.2:52065)
0926 20:45:52.763642 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:52.763672 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:52.763805 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:52.764221 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:52.764225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:52.764657 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:52.764954 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.764963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.766825 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.766829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.767809 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:52.767822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.768961 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.768975 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:52.769149 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.770024 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:52.770050 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:52.771357 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:52.775846 (+ 4489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.775849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.775852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.776221 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.776718 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:52.776722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:52.776724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:52.776905 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:52.777717 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:52.777721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:52.777942 (+ 221us) client_negotiation.cc:769] Sending connection context
0926 20:45:52.778059 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 20:45:52.778323 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":411,"threads_started":1}
I20260926 20:45:52.778963 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:52.762906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:52065 (local address 127.5.78.62:33839)
0926 20:45:52.763062 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:52.763066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:52.763782 (+ 716us) server_negotiation.cc:407] Connection header received
0926 20:45:52.763926 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:52.763935 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:52.764041 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:52.764170 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:52.765101 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.766636 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.767956 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:52.768707 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:52.768851 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:52.771589 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:52.771599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:52.771607 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:52.771658 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:52.775700 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.776387 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.776395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.776401 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.776579 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:52.777021 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:52.777029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:52.777034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:52.777574 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:52.777762 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:52.778203 (+ 441us) server_negotiation.cc:299] Negotiation successful
0926 20:45:52.778505 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:45:52.796104 6767 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:46029
I20260926 20:45:52.796149 6901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:46029 every 8 connection(s)
I20260926 20:45:52.797402 6767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:45:52.799599 6902 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33839
I20260926 20:45:52.799851 6902 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:52.800503 6902 heartbeater.cc:511] Master 127.5.78.62:33839 requested a full tablet report, sending...
I20260926 20:45:52.801973 6556 ts_manager.cc:194] Registered new tserver with Master: f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029)
I20260926 20:45:52.802855 6556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:60795
I20260926 20:45:52.806396 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6767
I20260926 20:45:52.806669 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:45:52.807184 5432 external_mini_cluster.cc:1428] Setting key 9a8b01d5235075cc7697b3e138831742
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:45:52.846755 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33839
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:53.185053 6906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:53.185434 6906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:53.185535 6906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:53.192845 6906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:53.192981 6906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:53.193064 6906 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:45:53.193145 6906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:45:53.203231 6906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:33839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:53.204522 6906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:53.205943 6906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:53.219880 6911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:53.219898 6912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:53.220147 6914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:53.220633 6906 server_base.cc:1034] running on GCE node
I20260926 20:45:53.221328 6906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:53.222286 6906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:53.223511 6906 hybrid_clock.cc:648] HybridClock initialized: now 1790455553223451 us; error 45 us; skew 500 ppm
I20260926 20:45:53.223845 6906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:53 dist-test-slave-z7j8 krb5kdc[6511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:53.227144 6906 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:45:53.228631 6906 webserver.cc:460] Webserver started at http://127.5.78.3:33101/ using document root <none> and password file <none>
I20260926 20:45:53.229314 6906 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:53.229399 6906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:53.229656 6906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:53.232507 6906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "96ffb7bf4c9a41a0a6d256531a815201"
format_stamp: "Formatted at 2026-09-26 20:45:53 on dist-test-slave-z7j8"
server_key: "8f988017e0fe6f1a490489312d44111d"
server_key_iv: "1f007c68141928023dea6f068239722f"
server_key_version: "encryptionkey@0"
I20260926 20:45:53.233222 6906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "96ffb7bf4c9a41a0a6d256531a815201"
format_stamp: "Formatted at 2026-09-26 20:45:53 on dist-test-slave-z7j8"
server_key: "8f988017e0fe6f1a490489312d44111d"
server_key_iv: "1f007c68141928023dea6f068239722f"
server_key_version: "encryptionkey@0"
I20260926 20:45:53.238299 6906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 20:45:53.241856 6921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:53.243361 6906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:53.243546 6906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "96ffb7bf4c9a41a0a6d256531a815201"
format_stamp: "Formatted at 2026-09-26 20:45:53 on dist-test-slave-z7j8"
server_key: "8f988017e0fe6f1a490489312d44111d"
server_key_iv: "1f007c68141928023dea6f068239722f"
server_key_version: "encryptionkey@0"
I20260926 20:45:53.243768 6906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:53.262490 6906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:53.266811 6763 heartbeater.cc:503] Master 127.5.78.62:33839 was elected leader, sending a full tablet report...
I20260926 20:45:53.266975 6906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:53.267342 6906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:53.270330 6906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:53.270470 6906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:53.270632 6906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:53.270756 6906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:45:53 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:53.297602 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.270632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:57965 (local address 127.5.78.62:33839)
0926 20:45:53.270840 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.270846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.272327 (+ 1481us) server_negotiation.cc:407] Connection header received
0926 20:45:53.275329 (+ 3002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.275337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.275433 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.275554 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:53.278536 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.280035 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.282996 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.283833 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.284018 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.288750 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:53.288757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:53.288761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:53.288802 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:53.293205 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:53.294125 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:53.294134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:53.294141 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:53.294356 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:53.295074 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:53.295080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:53.295092 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:53.295410 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:53.295586 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.296754 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.297078 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:45:53.298779 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.271248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.5.78.3:57965)
0926 20:45:53.272135 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.272200 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.274970 (+ 2770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.275859 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.275870 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.276868 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:53.278210 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.278259 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.280294 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.280300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.282793 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.282817 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.284097 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.284102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.284320 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.285136 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:53.285159 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:53.288392 (+ 3233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:53.293409 (+ 5017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:53.293422 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:53.293450 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:53.293960 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:53.294533 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:53.294542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:53.294548 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:53.294912 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:53.295574 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:53.295584 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:53.296127 (+ 543us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.296554 (+ 427us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.297197 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":347,"threads_started":1}
I20260926 20:45:53.320818 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.305178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:40403 (local address 127.5.78.62:33839)
0926 20:45:53.305418 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.305428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.306066 (+ 638us) server_negotiation.cc:407] Connection header received
0926 20:45:53.306303 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.306307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.306371 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.306548 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:53.308182 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.309121 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.310602 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.311263 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.311407 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.313934 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:53.313941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:53.313945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:53.313987 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:53.317633 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:53.318513 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:53.318518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:53.318522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:53.318684 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:53.319072 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:53.319077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:53.319080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:53.319454 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:53.319642 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.320204 (+ 562us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.320440 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:45:53.320902 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.305140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.5.78.3:40403)
0926 20:45:53.305925 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.305952 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.306141 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.306569 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.306574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.307441 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:53.308019 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.308031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.309292 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.309296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.310395 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.310468 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.311463 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.311470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.311665 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.312562 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:53.312579 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:53.313754 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:53.317776 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:53.317784 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:53.317789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:53.318364 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:53.318819 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:53.318822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:53.318825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:53.318946 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:53.319603 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:53.319613 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:53.319894 (+ 281us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.320074 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.320441 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":393,"threads_started":1}
I20260926 20:45:53.331799 6906 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:42765
I20260926 20:45:53.331822 7040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:42765 every 8 connection(s)
I20260926 20:45:53.333092 6906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:45:53.335357 7041 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33839
I20260926 20:45:53.335604 7041 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:53.336287 7041 heartbeater.cc:511] Master 127.5.78.62:33839 requested a full tablet report, sending...
I20260926 20:45:53.337679 6556 ts_manager.cc:194] Registered new tserver with Master: 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765)
I20260926 20:45:53.338125 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 6906
I20260926 20:45:53.338331 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:45:53.338649 6556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:57965
I20260926 20:45:53.338801 5432 external_mini_cluster.cc:1428] Setting key a5b2aa3dcad44530632ea31b076e3b37
I20260926 20:45:53.342113 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:45:53 dist-test-slave-z7j8 krb5kdc[6511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:45:53 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:53.389149 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.374837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.5.78.62:33839)
0926 20:45:53.375002 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.375011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.375380 (+ 369us) server_negotiation.cc:407] Connection header received
0926 20:45:53.375511 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.375515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.375581 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.375673 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:53.377142 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.378012 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.379455 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.379924 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.380029 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.383943 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:53.383949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:53.383952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:53.383987 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:53.386635 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:53.387343 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:53.387347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:53.387350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:53.387484 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:53.387879 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:53.387884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:53.387886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:53.388179 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:53.388364 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.388711 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.388883 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 20:45:53.393419 6556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:45:53.400098 6556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:53.422219 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.413059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.5.78.1:44399)
0926 20:45:53.413851 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.413860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.413890 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:45:53.414171 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.414178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.414368 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.414605 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.415361 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.416853 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.419506 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.420999 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.421119 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.421353 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.421746 (+ 393us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.421911 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":484,"threads_started":1}
I20260926 20:45:53.422339 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.412864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44399 (local address 127.0.0.1:46178)
0926 20:45:53.413756 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.413792 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.413939 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.414605 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.414612 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.414639 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.415115 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.415131 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.417012 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.417018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.419302 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.419319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.421192 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.421199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.421431 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.421513 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.421683 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.421910 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":24,"thread_start_us":1050,"threads_started":1}
I20260926 20:45:53.425817 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.414904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.5.78.2:46029)
0926 20:45:53.417856 (+ 2952us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.417862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.417885 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:45:53.417955 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.417960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.418191 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.419181 (+ 990us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.419388 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.420526 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.423373 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.424758 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.424913 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.425213 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.425358 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.425519 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2807,"thread_start_us":2678,"threads_started":1}
I20260926 20:45:53.425839 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.414672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:46029 (local address 127.0.0.1:40312)
0926 20:45:53.415316 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.415336 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.415432 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.418442 (+ 3010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.418446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.418472 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.418955 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.418966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.420665 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.420673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.423141 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.423164 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.424980 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.424986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.425150 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.425184 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.425280 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.425410 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":4,"thread_start_us":315,"threads_started":1}
I20260926 20:45:53.425881 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.415392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:42765 (local address 127.0.0.1:57556)
0926 20:45:53.416861 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.416878 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.416975 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.418192 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.418197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.418223 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.418802 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.418819 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.420996 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.421002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.423249 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.423263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.425066 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.425074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.425264 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.425300 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.425369 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.425497 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"thread_start_us":1228,"threads_started":1}
I20260926 20:45:53.426853 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.415942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57556 (local address 127.5.78.3:42765)
0926 20:45:53.417566 (+ 1624us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.417575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.417609 (+ 34us) server_negotiation.cc:407] Connection header received
0926 20:45:53.417704 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.417714 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.417988 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.418188 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.419214 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.420857 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.423418 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.424783 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.426158 (+ 1375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.426385 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.426477 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.426618 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1459,"thread_start_us":1271,"threads_started":1}
I20260926 20:45:53.426364 6693 tablet_service.cc:1467] Processing CreateTablet for tablet 8042420b93cb41b2b4dc48efed58afca (DEFAULT_TABLE table=test-table [id=36ca59e3a4934f3eb20b8d7cac07233c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:53.427867 6693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8042420b93cb41b2b4dc48efed58afca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:53.429862 6832 tablet_service.cc:1467] Processing CreateTablet for tablet 8042420b93cb41b2b4dc48efed58afca (DEFAULT_TABLE table=test-table [id=36ca59e3a4934f3eb20b8d7cac07233c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:53.430236 6971 tablet_service.cc:1467] Processing CreateTablet for tablet 8042420b93cb41b2b4dc48efed58afca (DEFAULT_TABLE table=test-table [id=36ca59e3a4934f3eb20b8d7cac07233c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:53.431140 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8042420b93cb41b2b4dc48efed58afca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:53.431685 6971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8042420b93cb41b2b4dc48efed58afca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:53.443235 7060 tablet_bootstrap.cc:492] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Bootstrap starting.
I20260926 20:45:53.446023 7060 tablet_bootstrap.cc:654] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:53.446964 7062 tablet_bootstrap.cc:492] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Bootstrap starting.
I20260926 20:45:53.447870 7060 log.cc:824] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:53.448688 7061 tablet_bootstrap.cc:492] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Bootstrap starting.
I20260926 20:45:53.451015 7062 tablet_bootstrap.cc:654] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:53.451350 7060 tablet_bootstrap.cc:492] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: No bootstrap required, opened a new log
I20260926 20:45:53.451692 7060 ts_tablet_manager.cc:1397] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260926 20:45:53.452301 7062 log.cc:824] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:53.452564 7061 tablet_bootstrap.cc:654] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:53.453725 7061 log.cc:824] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:53.455454 7062 tablet_bootstrap.cc:492] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: No bootstrap required, opened a new log
I20260926 20:45:53.455713 7062 ts_tablet_manager.cc:1397] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:45:53.456879 7061 tablet_bootstrap.cc:492] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: No bootstrap required, opened a new log
I20260926 20:45:53.457132 7061 ts_tablet_manager.cc:1397] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:45:53.462363 7060 raft_consensus.cc:348] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.462899 7060 raft_consensus.cc:374] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:53.463043 7060 raft_consensus.cc:729] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91d09ae3f954806995fb1e60a2cb4cd, State: Initialized, Role: FOLLOWER
I20260926 20:45:53.463600 7062 raft_consensus.cc:348] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.463701 7060 consensus_queue.cc:260] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.463933 7062 raft_consensus.cc:374] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:53.464049 7062 raft_consensus.cc:729] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ffb7bf4c9a41a0a6d256531a815201, State: Initialized, Role: FOLLOWER
I20260926 20:45:53.464624 7062 consensus_queue.cc:260] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.465685 7041 heartbeater.cc:503] Master 127.5.78.62:33839 was elected leader, sending a full tablet report...
I20260926 20:45:53.466286 7062 ts_tablet_manager.cc:1428] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260926 20:45:53.467660 7061 raft_consensus.cc:348] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.467996 7061 raft_consensus.cc:374] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:53.468113 7061 raft_consensus.cc:729] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a977f19a97be4494923ffd45f88b5bb4, State: Initialized, Role: FOLLOWER
I20260926 20:45:53.468662 7061 consensus_queue.cc:260] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.470268 7061 ts_tablet_manager.cc:1428] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260926 20:45:53.471174 6902 heartbeater.cc:503] Master 127.5.78.62:33839 was elected leader, sending a full tablet report...
I20260926 20:45:53.471952 7060 ts_tablet_manager.cc:1428] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Time spent starting tablet: real 0.020s user 0.013s sys 0.006s
W20260926 20:45:53.510774 6764 tablet.cc:2369] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:53.531960 7067 raft_consensus.cc:482] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:53.532303 7067 raft_consensus.cc:504] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.533967 7067 leader_election.cc:288] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1:44399), f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029)
I20260926 20:45:53.540985 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.534455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:50021 (local address 127.5.78.1:44399)
0926 20:45:53.534653 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.534661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.534701 (+ 40us) server_negotiation.cc:407] Connection header received
0926 20:45:53.534984 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.534989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.535079 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.535170 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.535799 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.536845 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.538896 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.540123 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.540219 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.540293 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.540665 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.540803 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:45:53.541036 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.534339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44399 (local address 127.5.78.3:50021)
0926 20:45:53.534608 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.534632 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.534777 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.535349 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.535354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.535382 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.535640 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.535651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.536991 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.536997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.538730 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.538741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.540275 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.540281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.540490 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.540527 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.540622 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.540759 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 20:45:53.542141 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.534943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:46591 (local address 127.5.78.2:46029)
0926 20:45:53.535163 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.535172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.535791 (+ 619us) server_negotiation.cc:407] Connection header received
0926 20:45:53.536036 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.536040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.536097 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.536168 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.536800 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.537942 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.540071 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.541291 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.541435 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.541552 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.541869 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.541967 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:45:53.542279 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.534786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:46029 (local address 127.5.78.3:46591)
0926 20:45:53.535692 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.535708 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.535894 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.536213 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.536218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.536241 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.536585 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.536595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.538083 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.538087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.539911 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.539930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.541507 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.541512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.541694 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.541737 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.541835 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.541957 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":564,"threads_started":1}
I20260926 20:45:53.542151 6718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8042420b93cb41b2b4dc48efed58afca" candidate_uuid: "96ffb7bf4c9a41a0a6d256531a815201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a977f19a97be4494923ffd45f88b5bb4" is_pre_election: true
I20260926 20:45:53.542757 6718 raft_consensus.cc:2454] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ffb7bf4c9a41a0a6d256531a815201 in term 0.
I20260926 20:45:53.543402 6924 leader_election.cc:302] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ffb7bf4c9a41a0a6d256531a815201, a977f19a97be4494923ffd45f88b5bb4; no voters:
I20260926 20:45:53.543376 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8042420b93cb41b2b4dc48efed58afca" candidate_uuid: "96ffb7bf4c9a41a0a6d256531a815201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" is_pre_election: true
I20260926 20:45:53.543779 7067 raft_consensus.cc:2781] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:53.543807 6857 raft_consensus.cc:2454] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ffb7bf4c9a41a0a6d256531a815201 in term 0.
I20260926 20:45:53.543921 7067 raft_consensus.cc:482] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:53.544042 7067 raft_consensus.cc:3037] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:53.546316 7067 raft_consensus.cc:504] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.547020 7067 leader_election.cc:288] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [CANDIDATE]: Term 1 election: Requested vote from peers a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1:44399), f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029)
I20260926 20:45:53.547492 6718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8042420b93cb41b2b4dc48efed58afca" candidate_uuid: "96ffb7bf4c9a41a0a6d256531a815201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a977f19a97be4494923ffd45f88b5bb4"
I20260926 20:45:53.547535 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8042420b93cb41b2b4dc48efed58afca" candidate_uuid: "96ffb7bf4c9a41a0a6d256531a815201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91d09ae3f954806995fb1e60a2cb4cd"
I20260926 20:45:53.547698 6718 raft_consensus.cc:3037] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:53.547724 6857 raft_consensus.cc:3037] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Advancing to term 1
W20260926 20:45:53.549846 6903 tablet.cc:2369] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:53.550110 6718 raft_consensus.cc:2454] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ffb7bf4c9a41a0a6d256531a815201 in term 1.
I20260926 20:45:53.550177 6857 raft_consensus.cc:2454] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ffb7bf4c9a41a0a6d256531a815201 in term 1.
I20260926 20:45:53.550648 6924 leader_election.cc:302] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ffb7bf4c9a41a0a6d256531a815201, a977f19a97be4494923ffd45f88b5bb4; no voters:
I20260926 20:45:53.550971 7067 raft_consensus.cc:2781] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:53.551409 7067 raft_consensus.cc:686] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 LEADER]: Becoming Leader. State: Replica: 96ffb7bf4c9a41a0a6d256531a815201, State: Running, Role: LEADER
I20260926 20:45:53.551885 7067 consensus_queue.cc:237] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.556821 6556 catalog_manager.cc:5495] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3). New cstate: current_term: 1 leader_uuid: "96ffb7bf4c9a41a0a6d256531a815201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } health_report { overall_health: HEALTHY } } }
W20260926 20:45:53.584754 7042 tablet.cc:2369] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:53.585824 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.579776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.5.78.3:42765)
0926 20:45:53.580016 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.580023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.580177 (+ 154us) server_negotiation.cc:407] Connection header received
0926 20:45:53.580585 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.580589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.580702 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.580871 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:53.581240 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.581953 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.583403 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.583858 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.584000 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.584553 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:53.585173 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:53.585339 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.585475 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.585559 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:45:53.594836 6718 raft_consensus.cc:1261] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Refusing update from remote peer 96ffb7bf4c9a41a0a6d256531a815201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:53.594863 6857 raft_consensus.cc:1261] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Refusing update from remote peer 96ffb7bf4c9a41a0a6d256531a815201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:53.595944 7067 consensus_queue.cc:1035] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:53.596271 7070 consensus_queue.cc:1035] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:53.631474 6556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:45:53.631757 6556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:45:53.636047 6556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: Sending DeleteTablet for 3 replicas of tablet 8042420b93cb41b2b4dc48efed58afca
I20260926 20:45:53.637861 6971 tablet_service.cc:1514] Processing DeleteTablet for tablet 8042420b93cb41b2b4dc48efed58afca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:53 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:57556
I20260926 20:45:53.638108 6832 tablet_service.cc:1514] Processing DeleteTablet for tablet 8042420b93cb41b2b4dc48efed58afca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:53 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40312
I20260926 20:45:53.638666 6693 tablet_service.cc:1514] Processing DeleteTablet for tablet 8042420b93cb41b2b4dc48efed58afca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:53 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:46178
I20260926 20:45:53.646533 6556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:45:53.648201 6556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:53.651005 7085 tablet_replica.cc:330] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: stopping tablet replica
I20260926 20:45:53.651582 7085 raft_consensus.cc:2229] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:45:53.652258 7083 tablet_replica.cc:330] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: stopping tablet replica
I20260926 20:45:53.652704 7083 raft_consensus.cc:2229] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:45:53.652894 7085 raft_consensus.cc:2258] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:53.653409 7083 raft_consensus.cc:2258] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:53.655645 7085 ts_tablet_manager.cc:1905] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:53.656513 6693 tablet_service.cc:1467] Processing CreateTablet for tablet b8f7b3b330a04cc495c4fda51f866e84 (DEFAULT_TABLE table=test-table [id=42bd716e534a4c78899549d3e18df13f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:53.657377 6831 tablet_service.cc:1467] Processing CreateTablet for tablet b8f7b3b330a04cc495c4fda51f866e84 (DEFAULT_TABLE table=test-table [id=42bd716e534a4c78899549d3e18df13f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:53.657568 6693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8f7b3b330a04cc495c4fda51f866e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:53.657963 6831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8f7b3b330a04cc495c4fda51f866e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:53.662995 6971 tablet_service.cc:1467] Processing CreateTablet for tablet b8f7b3b330a04cc495c4fda51f866e84 (DEFAULT_TABLE table=test-table [id=42bd716e534a4c78899549d3e18df13f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:53.663570 6971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8f7b3b330a04cc495c4fda51f866e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:53.664528 7061 tablet_bootstrap.cc:492] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Bootstrap starting.
I20260926 20:45:53.666330 7061 tablet_bootstrap.cc:654] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:53.666736 7084 tablet_replica.cc:330] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: stopping tablet replica
I20260926 20:45:53.667196 7084 raft_consensus.cc:2229] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:45:53.667721 7084 raft_consensus.cc:2258] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:53.668931 7060 tablet_bootstrap.cc:492] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Bootstrap starting.
I20260926 20:45:53.669759 7085 ts_tablet_manager.cc:1918] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:45:53.669894 7085 log.cc:1196] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/8042420b93cb41b2b4dc48efed58afca
I20260926 20:45:53.670070 7084 ts_tablet_manager.cc:1905] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:53.670560 7060 tablet_bootstrap.cc:654] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:53.670380 7085 ts_tablet_manager.cc:1939] T 8042420b93cb41b2b4dc48efed58afca P a977f19a97be4494923ffd45f88b5bb4: Deleting consensus metadata
I20260926 20:45:53.673915 6542 catalog_manager.cc:4841] TS a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1:44399): tablet 8042420b93cb41b2b4dc48efed58afca (table test-table [id=36ca59e3a4934f3eb20b8d7cac07233c]) successfully deleted
I20260926 20:45:53.678761 7084 ts_tablet_manager.cc:1918] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:45:53.679013 7084 log.cc:1196] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/8042420b93cb41b2b4dc48efed58afca
I20260926 20:45:53.679716 7084 ts_tablet_manager.cc:1939] T 8042420b93cb41b2b4dc48efed58afca P f91d09ae3f954806995fb1e60a2cb4cd: Deleting consensus metadata
I20260926 20:45:53.679971 7083 ts_tablet_manager.cc:1905] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:53.682024 6542 catalog_manager.cc:4841] TS f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029): tablet 8042420b93cb41b2b4dc48efed58afca (table test-table [id=36ca59e3a4934f3eb20b8d7cac07233c]) successfully deleted
I20260926 20:45:53.686053 7062 tablet_bootstrap.cc:492] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Bootstrap starting.
I20260926 20:45:53.687144 7061 tablet_bootstrap.cc:492] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: No bootstrap required, opened a new log
I20260926 20:45:53.687367 7061 ts_tablet_manager.cc:1397] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.004s
I20260926 20:45:53.687798 7060 tablet_bootstrap.cc:492] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: No bootstrap required, opened a new log
I20260926 20:45:53.687887 7062 tablet_bootstrap.cc:654] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:53.687953 7060 ts_tablet_manager.cc:1397] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.011s
I20260926 20:45:53.688517 7061 raft_consensus.cc:348] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.688810 7061 raft_consensus.cc:374] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:53.688808 7060 raft_consensus.cc:348] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.689051 7060 raft_consensus.cc:374] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:53.688987 7061 raft_consensus.cc:729] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a977f19a97be4494923ffd45f88b5bb4, State: Initialized, Role: FOLLOWER
I20260926 20:45:53.689160 7060 raft_consensus.cc:729] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91d09ae3f954806995fb1e60a2cb4cd, State: Initialized, Role: FOLLOWER
I20260926 20:45:53.689522 7060 consensus_queue.cc:260] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.689602 7061 consensus_queue.cc:260] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.692281 7061 ts_tablet_manager.cc:1428] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 20:45:53.696759 7060 ts_tablet_manager.cc:1428] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Time spent starting tablet: real 0.009s user 0.001s sys 0.002s
I20260926 20:45:53.698696 7083 ts_tablet_manager.cc:1918] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:45:53.699331 7083 log.cc:1196] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/8042420b93cb41b2b4dc48efed58afca
I20260926 20:45:53.700038 7083 ts_tablet_manager.cc:1939] T 8042420b93cb41b2b4dc48efed58afca P 96ffb7bf4c9a41a0a6d256531a815201: Deleting consensus metadata
I20260926 20:45:53.700377 7062 tablet_bootstrap.cc:492] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: No bootstrap required, opened a new log
I20260926 20:45:53.700547 7062 ts_tablet_manager.cc:1397] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260926 20:45:53.701937 6542 catalog_manager.cc:4841] TS 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765): tablet 8042420b93cb41b2b4dc48efed58afca (table test-table [id=36ca59e3a4934f3eb20b8d7cac07233c]) successfully deleted
I20260926 20:45:53.702034 7062 raft_consensus.cc:348] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.702279 7062 raft_consensus.cc:374] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:53.702466 7062 raft_consensus.cc:729] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ffb7bf4c9a41a0a6d256531a815201, State: Initialized, Role: FOLLOWER
I20260926 20:45:53.702754 7062 consensus_queue.cc:260] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.703338 7062 ts_tablet_manager.cc:1428] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 20:45:53.952382 7068 raft_consensus.cc:482] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:53.952661 7068 raft_consensus.cc:504] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.954447 7068 leader_election.cc:288] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029), 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765)
I20260926 20:45:53.964200 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.955794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:38625 (local address 127.5.78.3:42765)
0926 20:45:53.955980 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.955988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.956405 (+ 417us) server_negotiation.cc:407] Connection header received
0926 20:45:53.956547 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.956553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.956639 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.956742 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.957328 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.958235 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.961515 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.963009 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.963177 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.963280 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.963727 (+ 447us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.963888 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 20:45:53.964533 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.955621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:42765 (local address 127.5.78.1:38625)
0926 20:45:53.956281 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.956299 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.956430 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.956780 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.956785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.956811 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.957177 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.957185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.958406 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.958430 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.961276 (+ 2846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.961296 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.963295 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.963303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.963518 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.963558 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.963679 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.963835 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":223,"thread_start_us":355,"threads_started":1}
I20260926 20:45:53.964807 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.954877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:36937 (local address 127.5.78.2:46029)
0926 20:45:53.955057 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:53.955063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:53.955597 (+ 534us) server_negotiation.cc:407] Connection header received
0926 20:45:53.956071 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:53.956077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:53.956173 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:53.956245 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:53.957036 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.958178 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.962596 (+ 4418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.963583 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.963708 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.963800 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:53.964400 (+ 600us) server_negotiation.cc:299] Negotiation successful
0926 20:45:53.964542 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:45:53.965570 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:53.954766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:46029 (local address 127.5.78.1:36937)
0926 20:45:53.955490 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:53.955512 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:53.955709 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:53.956458 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:53.956466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:53.956494 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:53.956822 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.956836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.958334 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.958340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.961277 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:53.961296 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:53.963968 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:53.963976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:53.964169 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:53.964205 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:45:53.965072 (+ 867us) client_negotiation.cc:240] Negotiation successful
0926 20:45:53.965196 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":149,"thread_start_us":442,"threads_started":1}
I20260926 20:45:53.965724 6996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f7b3b330a04cc495c4fda51f866e84" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ffb7bf4c9a41a0a6d256531a815201" is_pre_election: true
I20260926 20:45:53.966207 6996 raft_consensus.cc:2454] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 0.
I20260926 20:45:53.966480 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f7b3b330a04cc495c4fda51f866e84" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" is_pre_election: true
I20260926 20:45:53.966744 6857 raft_consensus.cc:2454] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 0.
I20260926 20:45:53.966970 6645 leader_election.cc:302] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ffb7bf4c9a41a0a6d256531a815201, a977f19a97be4494923ffd45f88b5bb4; no voters:
I20260926 20:45:53.967446 7068 raft_consensus.cc:2781] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:53.967573 7068 raft_consensus.cc:482] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:53.967679 7068 raft_consensus.cc:3037] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:53.970331 7068 raft_consensus.cc:504] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.971163 7068 leader_election.cc:288] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 election: Requested vote from peers f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029), 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765)
I20260926 20:45:53.971686 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f7b3b330a04cc495c4fda51f866e84" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91d09ae3f954806995fb1e60a2cb4cd"
I20260926 20:45:53.971933 6857 raft_consensus.cc:3037] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:53.972111 6996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f7b3b330a04cc495c4fda51f866e84" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ffb7bf4c9a41a0a6d256531a815201"
I20260926 20:45:53.972319 6996 raft_consensus.cc:3037] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:53.974731 6996 raft_consensus.cc:2454] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 1.
I20260926 20:45:53.975020 6857 raft_consensus.cc:2454] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 1.
I20260926 20:45:53.975176 6645 leader_election.cc:302] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ffb7bf4c9a41a0a6d256531a815201, a977f19a97be4494923ffd45f88b5bb4; no voters:
I20260926 20:45:53.975632 7068 raft_consensus.cc:2781] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:53.976167 7068 raft_consensus.cc:686] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 1 LEADER]: Becoming Leader. State: Replica: a977f19a97be4494923ffd45f88b5bb4, State: Running, Role: LEADER
I20260926 20:45:53.976552 7068 consensus_queue.cc:237] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:53.980140 6555 catalog_manager.cc:5495] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1). New cstate: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } health_report { overall_health: UNKNOWN } } }
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455551, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:54.067919 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.052963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33839 (local address 127.0.0.1:52234)
0926 20:45:54.053999 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:54.054016 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:54.054147 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:54.054667 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:54.054671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:54.055291 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:54.055520 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:54.055528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.056898 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.056902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:54.057813 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:54.057823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.058816 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.058821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:54.058955 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.059749 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:54.059770 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:54.061991 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:54.065297 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:54.065325 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:54.065332 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:54.065794 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:54.066198 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:54.066202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:54.066204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:54.066348 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:54.066990 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:54.066997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:54.067162 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 20:45:54.067351 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 20:45:54.067583 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"thread_start_us":722,"threads_started":1}
I20260926 20:45:54.068065 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.052982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.5.78.62:33839)
0926 20:45:54.053667 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.053673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.054108 (+ 435us) server_negotiation.cc:407] Connection header received
0926 20:45:54.054258 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.054263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.054338 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.054552 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:54.055679 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.056757 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.057954 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.058590 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.058729 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.062204 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:54.062214 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:54.062222 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:54.062260 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:54.065143 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.065925 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.065929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.065931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.066055 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.066544 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.066547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.066550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.066818 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:54.066935 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.067414 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.067704 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":516,"thread_start_us":324,"threads_started":1}
I20260926 20:45:54.073551 6555 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:52234:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 20:45:54.074698 6555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:54.081525 6693 tablet_service.cc:1467] Processing CreateTablet for tablet af85d4f92f4b4d8da94ce042f34795b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=73ec781c842045c980644780bedfa4af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:45:54.081687 6831 tablet_service.cc:1467] Processing CreateTablet for tablet af85d4f92f4b4d8da94ce042f34795b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=73ec781c842045c980644780bedfa4af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:45:54.082093 6693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af85d4f92f4b4d8da94ce042f34795b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:54.082281 6831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af85d4f92f4b4d8da94ce042f34795b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:54.082825 6971 tablet_service.cc:1467] Processing CreateTablet for tablet af85d4f92f4b4d8da94ce042f34795b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=73ec781c842045c980644780bedfa4af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:45:54.083336 6971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af85d4f92f4b4d8da94ce042f34795b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:54.088459 7060 tablet_bootstrap.cc:492] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: Bootstrap starting.
I20260926 20:45:54.089154 7061 tablet_bootstrap.cc:492] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: Bootstrap starting.
I20260926 20:45:54.089979 7062 tablet_bootstrap.cc:492] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: Bootstrap starting.
I20260926 20:45:54.091003 7061 tablet_bootstrap.cc:654] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:54.091255 7060 tablet_bootstrap.cc:654] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:54.091675 7062 tablet_bootstrap.cc:654] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:54.093592 7060 tablet_bootstrap.cc:492] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: No bootstrap required, opened a new log
I20260926 20:45:54.093798 7060 ts_tablet_manager.cc:1397] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 20:45:54.093889 7062 tablet_bootstrap.cc:492] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: No bootstrap required, opened a new log
I20260926 20:45:54.094075 7062 ts_tablet_manager.cc:1397] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 20:45:54.094257 7061 tablet_bootstrap.cc:492] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: No bootstrap required, opened a new log
I20260926 20:45:54.094442 7061 ts_tablet_manager.cc:1397] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 20:45:54.094661 7060 raft_consensus.cc:348] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.094954 7060 raft_consensus.cc:374] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:54.094854 7062 raft_consensus.cc:348] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.095041 7060 raft_consensus.cc:729] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91d09ae3f954806995fb1e60a2cb4cd, State: Initialized, Role: FOLLOWER
I20260926 20:45:54.095074 7062 raft_consensus.cc:374] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:54.095165 7062 raft_consensus.cc:729] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ffb7bf4c9a41a0a6d256531a815201, State: Initialized, Role: FOLLOWER
I20260926 20:45:54.095130 7061 raft_consensus.cc:348] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.095373 7061 raft_consensus.cc:374] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:54.095456 7061 raft_consensus.cc:729] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a977f19a97be4494923ffd45f88b5bb4, State: Initialized, Role: FOLLOWER
I20260926 20:45:54.095323 7060 consensus_queue.cc:260] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.095427 7062 consensus_queue.cc:260] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.095950 7060 ts_tablet_manager.cc:1428] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:45:54.096000 7062 ts_tablet_manager.cc:1428] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:45:54.095837 7070 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.095795 7066 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.096145 7070 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.096150 7066 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.096752 7061 consensus_queue.cc:260] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.097235 7068 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.097478 7068 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.098971 7061 ts_tablet_manager.cc:1428] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 20:45:54.199638 7068 raft_consensus.cc:482] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:54.199892 7068 raft_consensus.cc:504] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.200618 7068 leader_election.cc:288] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029), 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765)
I20260926 20:45:54.201356 6996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af85d4f92f4b4d8da94ce042f34795b9" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ffb7bf4c9a41a0a6d256531a815201" is_pre_election: true
I20260926 20:45:54.201356 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af85d4f92f4b4d8da94ce042f34795b9" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" is_pre_election: true
I20260926 20:45:54.201689 6996 raft_consensus.cc:2454] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 0.
I20260926 20:45:54.201694 6857 raft_consensus.cc:2454] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 0.
I20260926 20:45:54.202219 6645 leader_election.cc:302] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ffb7bf4c9a41a0a6d256531a815201, a977f19a97be4494923ffd45f88b5bb4; no voters:
I20260926 20:45:54.202564 7068 raft_consensus.cc:2781] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:54.202672 7068 raft_consensus.cc:482] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:54.202795 7068 raft_consensus.cc:3037] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:54.204733 7068 raft_consensus.cc:504] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.205361 7068 leader_election.cc:288] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 election: Requested vote from peers f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029), 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765)
I20260926 20:45:54.206040 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af85d4f92f4b4d8da94ce042f34795b9" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91d09ae3f954806995fb1e60a2cb4cd"
I20260926 20:45:54.206040 6996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af85d4f92f4b4d8da94ce042f34795b9" candidate_uuid: "a977f19a97be4494923ffd45f88b5bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ffb7bf4c9a41a0a6d256531a815201"
I20260926 20:45:54.206270 6996 raft_consensus.cc:3037] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:54.206293 6857 raft_consensus.cc:3037] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:54.209239 6857 raft_consensus.cc:2454] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 1.
I20260926 20:45:54.209239 6996 raft_consensus.cc:2454] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a977f19a97be4494923ffd45f88b5bb4 in term 1.
I20260926 20:45:54.209764 6645 leader_election.cc:302] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a977f19a97be4494923ffd45f88b5bb4, f91d09ae3f954806995fb1e60a2cb4cd; no voters:
I20260926 20:45:54.210186 7068 raft_consensus.cc:2781] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:54.210377 7068 raft_consensus.cc:686] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [term 1 LEADER]: Becoming Leader. State: Replica: a977f19a97be4494923ffd45f88b5bb4, State: Running, Role: LEADER
I20260926 20:45:54.210701 7068 consensus_queue.cc:237] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } }
I20260926 20:45:54.212229 7091 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: TxnStatusTablet state changed. Reason: New leader a977f19a97be4494923ffd45f88b5bb4. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.212540 7091 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:45:54.213063 7104 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 20:45:54.213294 7104 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 20:45:54.213682 6555 catalog_manager.cc:5495] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1). New cstate: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } health_report { overall_health: UNKNOWN } } }
I20260926 20:45:54.261456 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.253296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44399 (local address 127.0.0.1:46192)
0926 20:45:54.253584 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:54.253611 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:54.253767 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:54.254628 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:54.254631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:54.254641 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:45:54.254790 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:54.254797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.256520 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.256528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:54.258139 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:54.258154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.259367 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.259373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:54.259593 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.259670 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:45:54.260884 (+ 1214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:45:54.260906 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 20:45:54.260986 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 20:45:54.261145 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 20:45:54.261989 7107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.253332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.5.78.1:44399)
0926 20:45:54.254083 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.254091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.254115 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:45:54.254247 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.254256 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.254393 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.254622 (+ 229us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:54.254964 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.256362 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.258288 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.259122 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.259344 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.259912 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:54.260733 (+ 821us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:54.260912 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.261026 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.261122 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":318,"threads_started":1}
I20260926 20:45:54.266831 6996 raft_consensus.cc:1261] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Refusing update from remote peer a977f19a97be4494923ffd45f88b5bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:54.266999 6857 raft_consensus.cc:1261] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Refusing update from remote peer a977f19a97be4494923ffd45f88b5bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:54.267798 7091 consensus_queue.cc:1035] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:54.268232 7068 consensus_queue.cc:1035] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:54.290112 7070 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: TxnStatusTablet state changed. Reason: New leader a977f19a97be4494923ffd45f88b5bb4. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.290434 7070 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.291237 7066 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: TxnStatusTablet state changed. Reason: New leader a977f19a97be4494923ffd45f88b5bb4. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.291572 7066 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.297341 7068 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.297734 7068 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:45:54.298029 7068 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.298260 7068 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P a977f19a97be4494923ffd45f88b5bb4: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:45:54.302223 7070 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.302597 7070 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P 96ffb7bf4c9a41a0a6d256531a815201: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.309325 7066 tablet_replica.cc:439] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a977f19a97be4494923ffd45f88b5bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a977f19a97be4494923ffd45f88b5bb4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44399 } } peers { permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 } } peers { permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 } } }
I20260926 20:45:54.309648 7066 tablet_replica.cc:442] T af85d4f92f4b4d8da94ce042f34795b9 P f91d09ae3f954806995fb1e60a2cb4cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:54.316826 7107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.309454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.5.78.1:44399)
0926 20:45:54.309629 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.309635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.310766 (+ 1131us) server_negotiation.cc:407] Connection header received
0926 20:45:54.311076 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.311083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.311179 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.311296 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:54.311868 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.312713 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.314799 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.315272 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.315387 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.315939 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:54.316061 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:54.316148 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.316365 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.316443 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:45:54.336711 7107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.329110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:34647 (local address 127.5.78.1:44399)
0926 20:45:54.329296 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.329300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.329949 (+ 649us) server_negotiation.cc:407] Connection header received
0926 20:45:54.330154 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.330159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.330254 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.330347 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:54.330895 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.332082 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.334394 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.335102 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.335235 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.335821 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:54.335960 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:54.336048 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.336232 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.336330 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 20:45:54.337532 7120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.329078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44399 (local address 127.5.78.1:34647)
0926 20:45:54.329826 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:54.329842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:54.330032 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:54.330362 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:54.330366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:54.330386 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:45:54.330752 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:54.330761 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.332228 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.332236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:54.334228 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:54.334247 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.335313 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.335320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:54.335524 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.335606 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:45:54.336067 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:45:54.336087 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 20:45:54.336926 (+ 839us) client_negotiation.cc:240] Negotiation successful
0926 20:45:54.337079 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":448,"threads_started":1}
I20260926 20:45:54.349911 6996 raft_consensus.cc:1261] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Refusing update from remote peer a977f19a97be4494923ffd45f88b5bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:54.350298 6857 raft_consensus.cc:1261] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Refusing update from remote peer a977f19a97be4494923ffd45f88b5bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:54.350482 7068 consensus_queue.cc:1035] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ffb7bf4c9a41a0a6d256531a815201" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 42765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:54.350916 7068 consensus_queue.cc:1035] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91d09ae3f954806995fb1e60a2cb4cd" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:54.366508 7108 mvcc.cc:205] Tried to move back new op lower bound from 7333705950612258816 to 7333705949091467264. Current Snapshot: MvccSnapshot[applied={T|T < 7333705950612258816}]
I20260926 20:45:54.372706 7109 mvcc.cc:205] Tried to move back new op lower bound from 7333705950612258816 to 7333705949091467264. Current Snapshot: MvccSnapshot[applied={T|T < 7333705950612258816}]
I20260926 20:45:54.430485 6556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:45:54.430809 6556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:45:54.434221 6556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f991b543dd4545b1a2843b9bfdc852e0: Sending DeleteTablet for 3 replicas of tablet b8f7b3b330a04cc495c4fda51f866e84
I20260926 20:45:54.435098 6693 tablet_service.cc:1514] Processing DeleteTablet for tablet b8f7b3b330a04cc495c4fda51f866e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:54 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:46178
I20260926 20:45:54.435238 6971 tablet_service.cc:1514] Processing DeleteTablet for tablet b8f7b3b330a04cc495c4fda51f866e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:54 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:57556
I20260926 20:45:54.435932 7126 tablet_replica.cc:330] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: stopping tablet replica
I20260926 20:45:54.435953 6831 tablet_service.cc:1514] Processing DeleteTablet for tablet b8f7b3b330a04cc495c4fda51f866e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:54 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40312
I20260926 20:45:54.436302 7126 raft_consensus.cc:2229] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:45:54.436646 7126 raft_consensus.cc:2258] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:54.437912 7126 ts_tablet_manager.cc:1905] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:54.446357 7125 tablet_replica.cc:330] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: stopping tablet replica
I20260926 20:45:54.446664 7125 raft_consensus.cc:2229] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:45:54.447453 7125 raft_consensus.cc:2258] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:54.448017 7126 ts_tablet_manager.cc:1918] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:45:54.448310 7126 log.cc:1196] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/b8f7b3b330a04cc495c4fda51f866e84
I20260926 20:45:54.448743 7125 ts_tablet_manager.cc:1905] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:54.449059 7126 ts_tablet_manager.cc:1939] T b8f7b3b330a04cc495c4fda51f866e84 P 96ffb7bf4c9a41a0a6d256531a815201: Deleting consensus metadata
I20260926 20:45:54.450512 6542 catalog_manager.cc:4841] TS 96ffb7bf4c9a41a0a6d256531a815201 (127.5.78.3:42765): tablet b8f7b3b330a04cc495c4fda51f866e84 (table test-table [id=42bd716e534a4c78899549d3e18df13f]) successfully deleted
I20260926 20:45:54.453909 7127 tablet_replica.cc:330] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: stopping tablet replica
I20260926 20:45:54.454303 7127 raft_consensus.cc:2229] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:45:54.454777 7127 raft_consensus.cc:2258] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:54.456212 7127 ts_tablet_manager.cc:1905] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:54.456983 7125 ts_tablet_manager.cc:1918] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:45:54.457147 7125 log.cc:1196] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/b8f7b3b330a04cc495c4fda51f866e84
I20260926 20:45:54.457675 7125 ts_tablet_manager.cc:1939] T b8f7b3b330a04cc495c4fda51f866e84 P a977f19a97be4494923ffd45f88b5bb4: Deleting consensus metadata
I20260926 20:45:54.458935 6542 catalog_manager.cc:4841] TS a977f19a97be4494923ffd45f88b5bb4 (127.5.78.1:44399): tablet b8f7b3b330a04cc495c4fda51f866e84 (table test-table [id=42bd716e534a4c78899549d3e18df13f]) successfully deleted
I20260926 20:45:54.463173 7127 ts_tablet_manager.cc:1918] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:45:54.463323 7127 log.cc:1196] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/b8f7b3b330a04cc495c4fda51f866e84
I20260926 20:45:54.463716 7127 ts_tablet_manager.cc:1939] T b8f7b3b330a04cc495c4fda51f866e84 P f91d09ae3f954806995fb1e60a2cb4cd: Deleting consensus metadata
I20260926 20:45:54.464821 6542 catalog_manager.cc:4841] TS f91d09ae3f954806995fb1e60a2cb4cd (127.5.78.2:46029): tablet b8f7b3b330a04cc495c4fda51f866e84 (table test-table [id=42bd716e534a4c78899549d3e18df13f]) successfully deleted
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:45:54.474963 7107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.454208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.5.78.1:44399)
0926 20:45:54.454427 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.454446 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.456436 (+ 1990us) server_negotiation.cc:407] Connection header received
0926 20:45:54.456925 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.456934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.457024 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.457115 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:54.459479 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.460570 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.462461 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.463256 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.463416 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.468374 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:54.468388 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:54.468404 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:54.468444 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:54.472050 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.472982 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.472989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.472992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.473162 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.473573 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.473576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.473578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.473913 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:54.474153 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.474553 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.474712 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260926 20:45:54.475809 6758 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46208
I20260926 20:45:54.494050 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.479290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.5.78.62:33839)
0926 20:45:54.479566 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.479576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.480032 (+ 456us) server_negotiation.cc:407] Connection header received
0926 20:45:54.480197 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.480203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.480291 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.480421 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:54.481741 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.482941 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.484144 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.484839 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.484996 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.487424 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:54.487433 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:54.487438 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:54.487486 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:54.491043 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.491843 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.491849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.491854 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.492024 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.492427 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.492432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.492436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.492813 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:54.493089 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.493393 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.493638 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 20:45:54.494728 6556 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52240
I20260926 20:45:54.497685 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6627
I20260926 20:45:54.511620 7107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.498602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46224 (local address 127.5.78.1:44399)
0926 20:45:54.498837 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.498851 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.499289 (+ 438us) server_negotiation.cc:407] Connection header received
0926 20:45:54.499486 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.499491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.499555 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.499668 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:54.500778 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.501630 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.502943 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.503395 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.503527 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.505716 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:54.505721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:54.505724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:54.505756 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:54.508561 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.509233 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.509240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.509244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.509404 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.509757 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.509763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.509767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.510085 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:54.510379 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.511217 (+ 838us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.511376 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 20:45:54.512154 6758 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46224
W20260926 20:45:54.512723 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6627: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:54.512948 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6627
I20260926 20:45:54.532491 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6767
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:45:54.548089 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.533277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.5.78.2:46029)
0926 20:45:54.533742 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.533747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.533763 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:45:54.533815 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.533819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.533883 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.533985 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:54.535177 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.535922 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.537253 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.537731 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.537838 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.542105 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:54.542121 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:54.542137 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:54.542177 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:54.545204 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.545921 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.545929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.545931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.546071 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.546577 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.546585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.546590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.546978 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:54.547199 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.547655 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.547828 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":277,"threads_started":1}
W20260926 20:45:54.548734 6897 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40314
W20260926 20:45:54.549340 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6767: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:54.549510 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6767
I20260926 20:45:54.565917 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6906
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[6511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:45:54.582860 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:54.566732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.5.78.3:42765)
0926 20:45:54.567281 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:54.567285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:54.567301 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:45:54.567354 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:54.567357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:54.567414 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:54.567514 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:54.568853 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.569673 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.571346 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:54.571852 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:54.572056 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:54.576411 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:54.576428 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:54.576444 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:54.576483 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:54.579900 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.580697 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.580705 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.580707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.580840 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:54.581266 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:54.581272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:54.581275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:54.581643 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:54.581954 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:54.582234 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 20:45:54.582501 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":308,"threads_started":1}
W20260926 20:45:54.583563 7036 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57582
W20260926 20:45:54.584223 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6906: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:54.584359 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6906
I20260926 20:45:54.605144 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 6524
I20260926 20:45:54.606258 6596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:54.719452 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 6524
2026-09-26T20:45:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4856 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[7138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[7138](info): set up 2 sockets
Sep 26 20:45:54 dist-test-slave-z7j8 krb5kdc[7138](info): commencing operation
krb5kdc: starting...
W20260926 20:45:55.802628 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:45:55 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:45:55.998394 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44623
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:56.328457 7151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:56.328847 7151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:56.329001 7151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:56.336658 7151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:56.336815 7151 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 20:45:56.336910 7151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:56.337024 7151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:56.337107 7151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:56.337196 7151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:56.347292 7151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44623
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:56.348659 7151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:56.350020 7151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:56.362485 7156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:56.362371 7157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:56.362399 7159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:56.363797 7151 server_base.cc:1034] running on GCE node
I20260926 20:45:56.364674 7151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:56.366205 7151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:56.367491 7151 hybrid_clock.cc:648] HybridClock initialized: now 1790455556367388 us; error 50 us; skew 500 ppm
I20260926 20:45:56.367918 7151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:56 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455556, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:56.375288 7151 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:45:56.377028 7151 webserver.cc:460] Webserver started at http://127.5.78.62:37981/ using document root <none> and password file <none>
I20260926 20:45:56.377802 7151 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:56.377935 7151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:56.378253 7151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:56.381395 7151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "5505554f10cc4a9fad28f3342ebebc35"
format_stamp: "Formatted at 2026-09-26 20:45:56 on dist-test-slave-z7j8"
server_key: "627b68acce19a2b25d6f10834de9600d"
server_key_iv: "4f83bd76e2d560dd0c24b77b02a5479f"
server_key_version: "encryptionkey@0"
I20260926 20:45:56.382239 7151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "5505554f10cc4a9fad28f3342ebebc35"
format_stamp: "Formatted at 2026-09-26 20:45:56 on dist-test-slave-z7j8"
server_key: "627b68acce19a2b25d6f10834de9600d"
server_key_iv: "4f83bd76e2d560dd0c24b77b02a5479f"
server_key_version: "encryptionkey@0"
I20260926 20:45:56.387696 7151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:45:56.391454 7166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:56.392995 7151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:45:56.393198 7151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "5505554f10cc4a9fad28f3342ebebc35"
format_stamp: "Formatted at 2026-09-26 20:45:56 on dist-test-slave-z7j8"
server_key: "627b68acce19a2b25d6f10834de9600d"
server_key_iv: "4f83bd76e2d560dd0c24b77b02a5479f"
server_key_version: "encryptionkey@0"
I20260926 20:45:56.393419 7151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:56.411006 7151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:56.415738 7151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:56.416044 7151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:56.438313 7151 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:44623
I20260926 20:45:56.438334 7227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:44623 every 8 connection(s)
I20260926 20:45:56.439826 7151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:56.444180 7228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:56.447491 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 7151
I20260926 20:45:56.447726 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:45:56.448583 5432 external_mini_cluster.cc:1428] Setting key 48514286e433889877453aa967c34a27
I20260926 20:45:56.458494 7228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Bootstrap starting.
I20260926 20:45:56.462992 7228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:56.464526 7228 log.cc:824] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:45:56 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:56.467985 7228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: No bootstrap required, opened a new log
I20260926 20:45:56.474041 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:56.450942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.5.78.62:44623)
0926 20:45:56.451805 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:56.451819 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:56.451865 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:45:56.453654 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:56.453679 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:56.454119 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:56.454600 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:56.456582 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:56.457910 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:56.459819 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:56.460569 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:56.460718 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:56.466364 (+ 5646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:56.466372 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:56.466390 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:56.466483 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:56.469607 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:56.470398 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:56.470454 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:56.470469 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:56.470662 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:56.471087 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:56.471091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:56.471093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:56.471683 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:56.472008 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:56.472492 (+ 484us) server_negotiation.cc:299] Negotiation successful
0926 20:45:56.472936 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":423,"threads_started":1}
I20260926 20:45:56.476543 7228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505554f10cc4a9fad28f3342ebebc35" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44623 } }
I20260926 20:45:56.476969 7228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:56.477087 7228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5505554f10cc4a9fad28f3342ebebc35, State: Initialized, Role: FOLLOWER
I20260926 20:45:56.477787 7228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505554f10cc4a9fad28f3342ebebc35" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44623 } }
I20260926 20:45:56.478055 7228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:56.478199 7228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:56.478382 7228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:56.481215 7228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505554f10cc4a9fad28f3342ebebc35" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44623 } }
I20260926 20:45:56.481832 7228 leader_election.cc:302] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5505554f10cc4a9fad28f3342ebebc35; no voters:
I20260926 20:45:56.482542 7228 leader_election.cc:288] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:56.482689 7233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:56.483505 7233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [term 1 LEADER]: Becoming Leader. State: Replica: 5505554f10cc4a9fad28f3342ebebc35, State: Running, Role: LEADER
I20260926 20:45:56.484062 7233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505554f10cc4a9fad28f3342ebebc35" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44623 } }
I20260926 20:45:56.484999 7228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:56.487035 7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5505554f10cc4a9fad28f3342ebebc35. Latest consensus state: current_term: 1 leader_uuid: "5505554f10cc4a9fad28f3342ebebc35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505554f10cc4a9fad28f3342ebebc35" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44623 } } }
I20260926 20:45:56.487397 7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:56.488302 7239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:56.488139 7234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5505554f10cc4a9fad28f3342ebebc35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505554f10cc4a9fad28f3342ebebc35" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44623 } } }
I20260926 20:45:56.489384 7234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:56.493554 7239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:56.504091 7239 catalog_manager.cc:1347] Generated new cluster ID: fe117f38e8144e7f9520e5117a60cf47
I20260926 20:45:56.504246 7239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:56.518831 7239 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:56.520797 7239 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:56.532860 7239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Generated new TSK 0
I20260926 20:45:56.533783 7239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:45:56.591193 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44623
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:56.927873 7254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:56.928167 7254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:56.928342 7254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:56.935679 7254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:56.935812 7254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:56.935957 7254 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 20:45:56.936031 7254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:45:56.936147 7254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:45:56.946069 7254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:44623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:56.947465 7254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:56.948855 7254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:56.963356 7259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:56.963439 7260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:56.963922 7262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:56.964160 7254 server_base.cc:1034] running on GCE node
I20260926 20:45:56.964774 7254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:56.965796 7254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:56.967046 7254 hybrid_clock.cc:648] HybridClock initialized: now 1790455556966996 us; error 55 us; skew 500 ppm
I20260926 20:45:56.967411 7254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:56 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455556, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:56.970765 7254 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:45:56.972265 7254 webserver.cc:460] Webserver started at http://127.5.78.1:33461/ using document root <none> and password file <none>
I20260926 20:45:56.972987 7254 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:56.973124 7254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:56.973439 7254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:56.976451 7254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "b5f80aaefe62404eb18d0ea7f943a108"
format_stamp: "Formatted at 2026-09-26 20:45:56 on dist-test-slave-z7j8"
server_key: "2ec46b8258fb6474a079d49021b20c02"
server_key_iv: "4dff1e197fc23fc3dcd39ae5c22595a5"
server_key_version: "encryptionkey@0"
I20260926 20:45:56.977223 7254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "b5f80aaefe62404eb18d0ea7f943a108"
format_stamp: "Formatted at 2026-09-26 20:45:56 on dist-test-slave-z7j8"
server_key: "2ec46b8258fb6474a079d49021b20c02"
server_key_iv: "4dff1e197fc23fc3dcd39ae5c22595a5"
server_key_version: "encryptionkey@0"
I20260926 20:45:56.982910 7254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 20:45:56.986838 7269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:56.988397 7254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:45:56.988596 7254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "b5f80aaefe62404eb18d0ea7f943a108"
format_stamp: "Formatted at 2026-09-26 20:45:56 on dist-test-slave-z7j8"
server_key: "2ec46b8258fb6474a079d49021b20c02"
server_key_iv: "4dff1e197fc23fc3dcd39ae5c22595a5"
server_key_version: "encryptionkey@0"
I20260926 20:45:56.988816 7254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:57.013015 7254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:57.017422 7254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:57.017711 7254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:57.020386 7254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:57.020519 7254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:57.020651 7254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:57.020764 7254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:45:57 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455556, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:57.047037 7279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.021596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.5.78.1:49373)
0926 20:45:57.022560 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:57.022616 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:57.024467 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:57.025235 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:57.025263 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:57.026074 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:57.026897 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.026989 (+ 92us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.028672 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.028679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.030858 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.030872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.032120 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.032124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.032433 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.033225 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:57.033264 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:57.036871 (+ 3607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:57.041895 (+ 5024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.041907 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.041932 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.042858 (+ 926us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.043337 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.043342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.043349 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.043566 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.044095 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:57.044104 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:57.044488 (+ 384us) client_negotiation.cc:769] Sending connection context
0926 20:45:57.044769 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 20:45:57.045394 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":385,"threads_started":1}
I20260926 20:45:57.047242 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.020839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:49373 (local address 127.5.78.62:44623)
0926 20:45:57.021695 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:57.021703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:57.022728 (+ 1025us) server_negotiation.cc:407] Connection header received
0926 20:45:57.024756 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:57.024762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:57.024844 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:57.024961 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:57.027175 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.028485 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.031022 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.031711 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.031872 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.037218 (+ 5346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:57.037227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:57.037232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:57.037280 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:57.041639 (+ 4359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.043042 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.043051 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.043056 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.043201 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.043686 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.043691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.043694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.043972 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:57.044120 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:57.046604 (+ 2484us) server_negotiation.cc:299] Negotiation successful
0926 20:45:57.046849 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":639,"threads_started":1}
I20260926 20:45:57.070223 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.053820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.5.78.1:50673)
0926 20:45:57.054518 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:57.054542 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:57.054728 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:57.055175 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:57.055180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:57.055628 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:57.055923 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.055932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.057514 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.057522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.059422 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.059438 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.060615 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.060626 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.060760 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.061534 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:57.061564 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:57.062648 (+ 1084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:57.066779 (+ 4131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.066787 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.066793 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.067375 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.067846 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.067851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.067854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.068018 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.068668 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:57.068675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:57.069043 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 20:45:57.069208 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 20:45:57.069687 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":352,"threads_started":1}
I20260926 20:45:57.070778 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.053793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:50673 (local address 127.5.78.62:44623)
0926 20:45:57.053958 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:57.053962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:57.054651 (+ 689us) server_negotiation.cc:407] Connection header received
0926 20:45:57.054885 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:57.054894 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:57.055000 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:57.055108 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:57.056070 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.057344 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.059593 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.060155 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.060302 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.062898 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:57.062908 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:57.062915 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:57.062956 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:57.066613 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.067503 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.067508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.067512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.067703 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.068159 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.068164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.068166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.068526 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:57.068678 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:57.069947 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0926 20:45:57.070245 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:45:57.079762 7254 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:43711
I20260926 20:45:57.079845 7389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:43711 every 8 connection(s)
I20260926 20:45:57.080992 7254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:45:57.082635 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 7254
I20260926 20:45:57.082792 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:45:57.083352 5432 external_mini_cluster.cc:1428] Setting key 04ee41a872d14e5e8a53feba0b982628
I20260926 20:45:57.084734 7390 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44623
I20260926 20:45:57.084980 7390 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:57.085640 7390 heartbeater.cc:511] Master 127.5.78.62:44623 requested a full tablet report, sending...
I20260926 20:45:57.087574 7183 ts_manager.cc:194] Registered new tserver with Master: b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711)
I20260926 20:45:57.089012 7183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:49373
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:45:57.124051 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44623
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:57.448431 7394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:57.448863 7394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:57.449013 7394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:57.460703 7394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:57.460852 7394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:57.460989 7394 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 20:45:57.461059 7394 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:45:57.461159 7394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:45:57.471307 7394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:44623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:57.472535 7394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:57.473819 7394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:57.486732 7402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:57.486796 7399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:57.486819 7400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:57.487874 7394 server_base.cc:1034] running on GCE node
I20260926 20:45:57.488395 7394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:57.489233 7394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:57.490465 7394 hybrid_clock.cc:648] HybridClock initialized: now 1790455557490390 us; error 34 us; skew 500 ppm
I20260926 20:45:57.490789 7394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:57 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455557, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:57.493947 7394 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:45:57.495311 7394 webserver.cc:460] Webserver started at http://127.5.78.2:39327/ using document root <none> and password file <none>
I20260926 20:45:57.495913 7394 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:57.496006 7394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:57.496248 7394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:57.499281 7394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "5455b4187b0c4c92971999dcbf4a3246"
format_stamp: "Formatted at 2026-09-26 20:45:57 on dist-test-slave-z7j8"
server_key: "1f1321e47165362e184bb4f6977866e6"
server_key_iv: "d21605abcad2916c349ab62719770cc8"
server_key_version: "encryptionkey@0"
I20260926 20:45:57.500001 7394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "5455b4187b0c4c92971999dcbf4a3246"
format_stamp: "Formatted at 2026-09-26 20:45:57 on dist-test-slave-z7j8"
server_key: "1f1321e47165362e184bb4f6977866e6"
server_key_iv: "d21605abcad2916c349ab62719770cc8"
server_key_version: "encryptionkey@0"
I20260926 20:45:57.504928 7394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:57.508136 7409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:57.509347 7394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 20:45:57.509470 7394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "5455b4187b0c4c92971999dcbf4a3246"
format_stamp: "Formatted at 2026-09-26 20:45:57 on dist-test-slave-z7j8"
server_key: "1f1321e47165362e184bb4f6977866e6"
server_key_iv: "d21605abcad2916c349ab62719770cc8"
server_key_version: "encryptionkey@0"
I20260926 20:45:57.509635 7394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:57.521267 7394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:57.525314 7394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:57.525547 7394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:57.527976 7394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:57.528092 7394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:57.528214 7394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:57.528293 7394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:45:57 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455557, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:57.553903 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.528834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:35375 (local address 127.5.78.62:44623)
0926 20:45:57.529027 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:57.529032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:57.530460 (+ 1428us) server_negotiation.cc:407] Connection header received
0926 20:45:57.532907 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:57.532914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:57.533031 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:57.533131 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:57.535895 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.536964 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.539541 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.540336 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.540492 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.545929 (+ 5437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:57.545935 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:57.545938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:57.545975 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:57.549524 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.550655 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.550662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.550666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.550839 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.551518 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.551525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.551531 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.551937 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:57.552129 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:57.553249 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0926 20:45:57.553484 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:45:57.555676 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.529437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.5.78.2:35375)
0926 20:45:57.530280 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:57.530322 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:57.532503 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:57.533813 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:57.533833 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:57.534709 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:57.535691 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.535720 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.537126 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.537137 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.539345 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.539366 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.540678 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.540691 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.540926 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.541874 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:57.541897 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:57.545625 (+ 3728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:57.549699 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.549709 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.549733 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.550529 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.551018 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.551030 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.551036 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.551306 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.552219 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:57.552230 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:57.552747 (+ 517us) client_negotiation.cc:769] Sending connection context
0926 20:45:57.553154 (+ 407us) client_negotiation.cc:240] Negotiation successful
0926 20:45:57.553780 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":373,"threads_started":1}
I20260926 20:45:57.575223 7394 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:43613
I20260926 20:45:57.575234 7528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:43613 every 8 connection(s)
I20260926 20:45:57.576896 7394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:45:57.579012 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.562928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.5.78.2:58363)
0926 20:45:57.563700 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:57.563724 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:57.563897 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:57.564277 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:57.564284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:57.565018 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:57.565493 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.565511 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.566615 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.566619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.568327 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:57.568350 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.569388 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.569398 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:57.569670 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.570447 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:57.570477 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:57.571589 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:57.575658 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.575662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.575665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.576076 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.576498 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:57.576502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:57.576504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:57.576627 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:57.577715 (+ 1088us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:57.577720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:57.577971 (+ 251us) client_negotiation.cc:769] Sending connection context
0926 20:45:57.578162 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 20:45:57.578563 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":393,"threads_started":1}
I20260926 20:45:57.579243 7529 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44623
I20260926 20:45:57.579492 7529 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:57.579871 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:57.562940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:58363 (local address 127.5.78.62:44623)
0926 20:45:57.563143 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:57.563150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:57.563835 (+ 685us) server_negotiation.cc:407] Connection header received
0926 20:45:57.564062 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:57.564066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:57.564129 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:57.564219 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:57.565673 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.566392 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.568498 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:57.569130 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:57.569305 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:57.571778 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:57.571788 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:57.571796 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:57.571847 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:57.575527 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.576199 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.576206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.576213 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.576382 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:57.576731 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:57.576738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:57.576744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:57.577176 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:57.577345 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:57.579307 (+ 1962us) server_negotiation.cc:299] Negotiation successful
0926 20:45:57.579572 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:45:57.580384 7529 heartbeater.cc:511] Master 127.5.78.62:44623 requested a full tablet report, sending...
I20260926 20:45:57.581861 7183 ts_manager.cc:194] Registered new tserver with Master: 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613)
I20260926 20:45:57.582196 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 7394
I20260926 20:45:57.582345 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:45:57.582930 5432 external_mini_cluster.cc:1428] Setting key 35390bce5b4f1c0432619edcbd524ccc
I20260926 20:45:57.583040 7183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:35375
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:45:57.626724 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44623
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:57.955754 7533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:57.956219 7533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:57.956421 7533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:57.965372 7533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:57.965517 7533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:57.965624 7533 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 20:45:57.965720 7533 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:45:57.965888 7533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:45:57.977284 7533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:44623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:57.978801 7533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:57.980286 7533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:57.994464 7538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:57.994493 7539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:57.994540 7541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:57.995352 7533 server_base.cc:1034] running on GCE node
I20260926 20:45:57.996033 7533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:57.997056 7533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:57.998364 7533 hybrid_clock.cc:648] HybridClock initialized: now 1790455557998308 us; error 49 us; skew 500 ppm
I20260926 20:45:57.998769 7533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:57 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455557, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:58.001930 7533 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:45:58.003434 7533 webserver.cc:460] Webserver started at http://127.5.78.3:38491/ using document root <none> and password file <none>
I20260926 20:45:58.004189 7533 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:58.004293 7533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:58.004539 7533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:58.007625 7533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "2776beeb947341478cf5bd36226ee61b"
format_stamp: "Formatted at 2026-09-26 20:45:58 on dist-test-slave-z7j8"
server_key: "5aa8e3c362b51ab1635a1f652354c704"
server_key_iv: "327b6c0f8dde0b2fc909880faee90d19"
server_key_version: "encryptionkey@0"
I20260926 20:45:58.008401 7533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "2776beeb947341478cf5bd36226ee61b"
format_stamp: "Formatted at 2026-09-26 20:45:58 on dist-test-slave-z7j8"
server_key: "5aa8e3c362b51ab1635a1f652354c704"
server_key_iv: "327b6c0f8dde0b2fc909880faee90d19"
server_key_version: "encryptionkey@0"
I20260926 20:45:58.014006 7533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:45:58.017848 7548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:58.019613 7533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:58.019820 7533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "2776beeb947341478cf5bd36226ee61b"
format_stamp: "Formatted at 2026-09-26 20:45:58 on dist-test-slave-z7j8"
server_key: "5aa8e3c362b51ab1635a1f652354c704"
server_key_iv: "327b6c0f8dde0b2fc909880faee90d19"
server_key_version: "encryptionkey@0"
I20260926 20:45:58.020071 7533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:58.034065 7533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:58.038318 7533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:58.038671 7533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:58.041282 7533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:45:58.041409 7533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:58.041539 7533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:45:58.041640 7533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:45:58 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455557, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:58.067651 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.042301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:35779 (local address 127.5.78.62:44623)
0926 20:45:58.042553 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.042558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.044087 (+ 1529us) server_negotiation.cc:407] Connection header received
0926 20:45:58.046843 (+ 2756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.046851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.046945 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.047027 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:58.049710 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.050886 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.053662 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.054494 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.054680 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.059738 (+ 5058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:58.059749 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:58.059756 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:58.059813 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:58.063649 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.064761 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.064771 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.064777 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.064962 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.065463 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.065468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.065472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.065880 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:58.066036 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.067071 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.067294 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260926 20:45:58.069352 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.042963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.5.78.3:35779)
0926 20:45:58.043898 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.043938 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.046401 (+ 2463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.047623 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.047631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.048596 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:58.049524 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.049551 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.051064 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.051084 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.053366 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.053438 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.054778 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.054788 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.055109 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.056014 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:58.056039 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:58.059443 (+ 3404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:58.063845 (+ 4402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:58.063852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:58.063871 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:58.064599 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:58.065103 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:58.065107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:58.065109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:58.065323 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:58.066030 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:58.066038 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:58.066523 (+ 485us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.066947 (+ 424us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.067567 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":418,"threads_started":1}
I20260926 20:45:58.087096 7533 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:36821
I20260926 20:45:58.087146 7667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:36821 every 8 connection(s)
I20260926 20:45:58.088651 7533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:45:58.090870 7668 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44623
I20260926 20:45:58.091102 7668 heartbeater.cc:464] Registering TS with master...
I20260926 20:45:58.091737 7668 heartbeater.cc:511] Master 127.5.78.62:44623 requested a full tablet report, sending...
I20260926 20:45:58.091997 7390 heartbeater.cc:503] Master 127.5.78.62:44623 was elected leader, sending a full tablet report...
I20260926 20:45:58.093273 7182 ts_manager.cc:194] Registered new tserver with Master: 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:45:58.094384 7182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:35779
I20260926 20:45:58.095942 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.077918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:44501 (local address 127.5.78.62:44623)
0926 20:45:58.078122 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.078129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.079102 (+ 973us) server_negotiation.cc:407] Connection header received
0926 20:45:58.079373 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.079381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.079491 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.079560 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:58.084071 (+ 4511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.085133 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.086875 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.087465 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.087615 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.089702 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:58.089708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:58.089713 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:58.089751 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:58.093159 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.093945 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.093952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.093956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.094103 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.094460 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.094465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.094467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.094792 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:58.094950 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.095361 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.095586 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:45:58.096056 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.077997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.5.78.3:44501)
0926 20:45:58.078993 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.079014 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.079170 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.079740 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.079743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.083190 (+ 3447us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:58.083879 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.083899 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.085266 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.085271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.086661 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.086683 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.087655 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.087661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.087853 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.088511 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:58.088527 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:58.089433 (+ 906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:58.093295 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:58.093300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:58.093304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:58.093807 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:58.094205 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:58.094208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:58.094210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:58.094325 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:58.094913 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:58.094918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:58.095110 (+ 192us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.095242 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.095588 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":641,"threads_started":1}
I20260926 20:45:58.096184 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 7533
I20260926 20:45:58.096335 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:45:58.096849 5432 external_mini_cluster.cc:1428] Setting key 7082c9e9489f309b4970354f097eed2e
I20260926 20:45:58.099826 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:45:58 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:45:58 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:58.147990 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.133696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.5.78.62:44623)
0926 20:45:58.133862 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.133868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.134196 (+ 328us) server_negotiation.cc:407] Connection header received
0926 20:45:58.134361 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.134364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.134488 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.134589 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:58.136131 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.136912 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.138363 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.138872 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.139001 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.143047 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:58.143052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:58.143056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:58.143093 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:58.145519 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.146219 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.146223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.146226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.146358 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.146761 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.146764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.146766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.147031 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:58.147209 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.147533 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.147715 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:45:58.152724 7182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:45:58.160050 7182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:58.182163 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.172358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.5.78.1:43711)
0926 20:45:58.172926 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.172939 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.173772 (+ 833us) server_negotiation.cc:407] Connection header received
0926 20:45:58.174277 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.174305 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.174613 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.174801 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.175601 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.176896 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.179693 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.180994 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.181150 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.181378 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.181679 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.181840 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":600,"threads_started":1}
I20260926 20:45:58.182281 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.172193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43711 (local address 127.0.0.1:35716)
0926 20:45:58.173580 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.173629 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.173962 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.174924 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.174931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.174955 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.175353 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.175368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.177016 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.177022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.179474 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.179489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.181203 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.181209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.181427 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.181470 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.181628 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.181830 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":1,"thread_start_us":1067,"threads_started":1}
I20260926 20:45:58.183224 7683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.172761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:36821 (local address 127.0.0.1:40100)
0926 20:45:58.174202 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.174221 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.174343 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.175261 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.175270 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.175300 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.175683 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.175695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.178008 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.178014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.180241 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.180256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.182632 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.182639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.182809 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.182844 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.182901 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.182996 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1328,"thread_start_us":1173,"threads_started":1}
I20260926 20:45:58.184278 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.173060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.5.78.3:36821)
0926 20:45:58.174674 (+ 1614us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.174682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.174712 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:45:58.174805 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.174815 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.175055 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.175278 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.176339 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.177860 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.180455 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.182018 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.182157 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.183705 (+ 1548us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.183792 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.183971 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":1750,"threads_started":1}
I20260926 20:45:58.185415 7685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.174152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.0.0.1:48362)
0926 20:45:58.174661 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.174677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.174779 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.178173 (+ 3394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.178178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.178210 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.178733 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.178745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.180524 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.180530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.182647 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.182659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.184663 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.184670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.184872 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.184903 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.184981 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.185106 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":828,"threads_started":1}
I20260926 20:45:58.186029 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.174651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.5.78.2:43613)
0926 20:45:58.177471 (+ 2820us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.177480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.177507 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:45:58.177693 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.177702 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.177936 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.178187 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.178950 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.180390 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.182810 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.184148 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.184283 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.184494 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.185539 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.185759 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2661,"thread_start_us":2543,"threads_started":1}
I20260926 20:45:58.186712 7320 tablet_service.cc:1467] Processing CreateTablet for tablet 106eb90e180743a09ecc287b613e3006 (DEFAULT_TABLE table=test-table [id=d577895002e34da2b04daa80dc7828d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:58.188220 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106eb90e180743a09ecc287b613e3006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.187763 7598 tablet_service.cc:1467] Processing CreateTablet for tablet 106eb90e180743a09ecc287b613e3006 (DEFAULT_TABLE table=test-table [id=d577895002e34da2b04daa80dc7828d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:58.189265 7598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106eb90e180743a09ecc287b613e3006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.189745 7459 tablet_service.cc:1467] Processing CreateTablet for tablet 106eb90e180743a09ecc287b613e3006 (DEFAULT_TABLE table=test-table [id=d577895002e34da2b04daa80dc7828d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:58.191071 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106eb90e180743a09ecc287b613e3006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.200261 7687 tablet_bootstrap.cc:492] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Bootstrap starting.
I20260926 20:45:58.203888 7687 tablet_bootstrap.cc:654] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.204984 7687 log.cc:824] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:58.205717 7689 tablet_bootstrap.cc:492] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Bootstrap starting.
I20260926 20:45:58.206737 7688 tablet_bootstrap.cc:492] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Bootstrap starting.
I20260926 20:45:58.209595 7688 tablet_bootstrap.cc:654] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.210135 7689 tablet_bootstrap.cc:654] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.210928 7688 log.cc:824] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:58.211668 7689 log.cc:824] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:58.212913 7687 tablet_bootstrap.cc:492] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: No bootstrap required, opened a new log
I20260926 20:45:58.213311 7687 ts_tablet_manager.cc:1397] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260926 20:45:58.215231 7689 tablet_bootstrap.cc:492] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: No bootstrap required, opened a new log
I20260926 20:45:58.215502 7689 ts_tablet_manager.cc:1397] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260926 20:45:58.215514 7688 tablet_bootstrap.cc:492] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: No bootstrap required, opened a new log
I20260926 20:45:58.215770 7688 ts_tablet_manager.cc:1397] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260926 20:45:58.223024 7689 raft_consensus.cc:348] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.223609 7689 raft_consensus.cc:374] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.223726 7689 raft_consensus.cc:729] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.224247 7687 raft_consensus.cc:348] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.224431 7689 consensus_queue.cc:260] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.224720 7687 raft_consensus.cc:374] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.224817 7687 raft_consensus.cc:729] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.225400 7687 consensus_queue.cc:260] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.226434 7688 raft_consensus.cc:348] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.226853 7688 raft_consensus.cc:374] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.226943 7688 raft_consensus.cc:729] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776beeb947341478cf5bd36226ee61b, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.227404 7687 ts_tablet_manager.cc:1428] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 20:45:58.227632 7688 consensus_queue.cc:260] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.228685 7668 heartbeater.cc:503] Master 127.5.78.62:44623 was elected leader, sending a full tablet report...
I20260926 20:45:58.229408 7688 ts_tablet_manager.cc:1428] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260926 20:45:58.230321 7529 heartbeater.cc:503] Master 127.5.78.62:44623 was elected leader, sending a full tablet report...
I20260926 20:45:58.231101 7689 ts_tablet_manager.cc:1428] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
I20260926 20:45:58.326740 7693 raft_consensus.cc:482] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:58.327102 7693 raft_consensus.cc:504] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.328895 7693 leader_election.cc:288] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
W20260926 20:45:58.329388 7530 tablet.cc:2369] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:45:58.333750 7391 tablet.cc:2369] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:58.336302 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.329314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:39277 (local address 127.5.78.1:43711)
0926 20:45:58.329509 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.329515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.330381 (+ 866us) server_negotiation.cc:407] Connection header received
0926 20:45:58.330789 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.330794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.330867 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.330984 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.331588 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.332529 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.334590 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.335352 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.335509 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.335624 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.335921 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.336060 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:45:58.336498 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.329244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43711 (local address 127.5.78.2:39277)
0926 20:45:58.330256 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.330277 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.330608 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.331006 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.331010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.331037 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.331377 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.331385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.332692 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.332698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.334355 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.334368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.335611 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.335617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.335757 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.335782 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.335885 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.336028 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":2,"thread_start_us":1188,"threads_started":1}
I20260926 20:45:58.336782 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.329494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:47065 (local address 127.5.78.3:36821)
0926 20:45:58.329683 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.329691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.330925 (+ 1234us) server_negotiation.cc:407] Connection header received
0926 20:45:58.331107 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.331112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.331181 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.331281 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.331973 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.332797 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.335049 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.335838 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.336005 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.336115 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.336388 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.336528 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 20:45:58.336915 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.329237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:36821 (local address 127.5.78.2:47065)
0926 20:45:58.330816 (+ 1579us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.330837 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.330971 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.331300 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.331303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.331335 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.331785 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.331797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.332962 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.332966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.334890 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.334903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.336066 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.336072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.336233 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.336256 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.336361 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.336518 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"mutex_wait_us":329,"thread_start_us":339,"threads_started":1}
I20260926 20:45:58.337738 7341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106eb90e180743a09ecc287b613e3006" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f80aaefe62404eb18d0ea7f943a108" is_pre_election: true
I20260926 20:45:58.337980 7618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106eb90e180743a09ecc287b613e3006" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b" is_pre_election: true
I20260926 20:45:58.338266 7341 raft_consensus.cc:2454] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 0.
I20260926 20:45:58.338554 7618 raft_consensus.cc:2454] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 0.
I20260926 20:45:58.339005 7410 leader_election.cc:302] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:45:58.339383 7693 raft_consensus.cc:2781] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:58.339538 7693 raft_consensus.cc:482] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:58.339673 7693 raft_consensus.cc:3037] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Advancing to term 1
W20260926 20:45:58.340304 7669 tablet.cc:2369] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:58.341887 7693 raft_consensus.cc:504] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.342561 7693 leader_election.cc:288] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 election: Requested vote from peers b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:45:58.343163 7341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106eb90e180743a09ecc287b613e3006" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f80aaefe62404eb18d0ea7f943a108"
I20260926 20:45:58.343161 7618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106eb90e180743a09ecc287b613e3006" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b"
I20260926 20:45:58.343393 7618 raft_consensus.cc:3037] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:58.343393 7341 raft_consensus.cc:3037] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:58.346684 7341 raft_consensus.cc:2454] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 1.
I20260926 20:45:58.346690 7618 raft_consensus.cc:2454] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 1.
I20260926 20:45:58.347216 7410 leader_election.cc:302] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2776beeb947341478cf5bd36226ee61b, 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:45:58.347602 7693 raft_consensus.cc:2781] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:58.348054 7693 raft_consensus.cc:686] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 LEADER]: Becoming Leader. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Running, Role: LEADER
I20260926 20:45:58.348536 7693 consensus_queue.cc:237] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } }
I20260926 20:45:58.353729 7181 catalog_manager.cc:5495] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } health_report { overall_health: HEALTHY } } }
Sep 26 20:45:58 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455556, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:45:58.406675 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.392906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.5.78.62:44623)
0926 20:45:58.393108 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.393112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.393737 (+ 625us) server_negotiation.cc:407] Connection header received
0926 20:45:58.393932 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.393937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.394009 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.394107 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:58.395304 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.396033 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.397414 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.397822 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.398000 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.401282 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:58.401287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:58.401291 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:58.401319 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:58.403908 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.404620 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.404624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.404626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.404746 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:58.405198 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:58.405201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:58.405203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:58.405453 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:58.405570 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.406129 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.406293 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 20:45:58.406711 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.392906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44623 (local address 127.0.0.1:55388)
0926 20:45:58.393615 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.393648 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.393815 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.394134 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.394137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.394908 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:58.395165 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.395175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.396156 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.396164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.397286 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.397299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.398022 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.398028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.398207 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.398883 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:58.398901 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:58.401085 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:58.404055 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:58.404067 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:58.404073 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:58.404508 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:58.404932 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:58.404935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:58.404938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:58.405081 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:58.405609 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:58.405617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:58.405779 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.405949 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.406186 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":418,"threads_started":1}
I20260926 20:45:58.412551 7181 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:55388:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 20:45:58.413933 7181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:58.421393 7320 tablet_service.cc:1467] Processing CreateTablet for tablet a10626b5a8cb45078a3de53897c0de32 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7e3d24d1953844be8c4876ae155c5231]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:45:58.421416 7598 tablet_service.cc:1467] Processing CreateTablet for tablet a10626b5a8cb45078a3de53897c0de32 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7e3d24d1953844be8c4876ae155c5231]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:45:58.421393 7459 tablet_service.cc:1467] Processing CreateTablet for tablet a10626b5a8cb45078a3de53897c0de32 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7e3d24d1953844be8c4876ae155c5231]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:45:58.422457 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a10626b5a8cb45078a3de53897c0de32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.422459 7598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a10626b5a8cb45078a3de53897c0de32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.422639 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a10626b5a8cb45078a3de53897c0de32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.428972 7689 tablet_bootstrap.cc:492] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: Bootstrap starting.
I20260926 20:45:58.430078 7688 tablet_bootstrap.cc:492] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: Bootstrap starting.
I20260926 20:45:58.430092 7687 tablet_bootstrap.cc:492] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: Bootstrap starting.
I20260926 20:45:58.430819 7689 tablet_bootstrap.cc:654] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.431757 7687 tablet_bootstrap.cc:654] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.431893 7688 tablet_bootstrap.cc:654] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.433959 7689 tablet_bootstrap.cc:492] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: No bootstrap required, opened a new log
I20260926 20:45:58.434127 7689 ts_tablet_manager.cc:1397] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 20:45:58.434302 7688 tablet_bootstrap.cc:492] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: No bootstrap required, opened a new log
I20260926 20:45:58.434507 7688 ts_tablet_manager.cc:1397] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 20:45:58.435043 7687 tablet_bootstrap.cc:492] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: No bootstrap required, opened a new log
I20260926 20:45:58.434942 7689 raft_consensus.cc:348] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.435155 7689 raft_consensus.cc:374] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.435205 7687 ts_tablet_manager.cc:1397] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 20:45:58.435250 7689 raft_consensus.cc:729] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.435514 7689 consensus_queue.cc:260] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.435410 7688 raft_consensus.cc:348] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.435695 7688 raft_consensus.cc:374] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.435804 7688 raft_consensus.cc:729] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776beeb947341478cf5bd36226ee61b, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.435904 7693 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.436209 7693 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.436072 7688 consensus_queue.cc:260] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.436080 7687 raft_consensus.cc:348] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.436779 7688 ts_tablet_manager.cc:1428] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 20:45:58.436582 7695 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.437003 7695 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.436326 7687 raft_consensus.cc:374] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.438242 7687 raft_consensus.cc:729] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.438616 7687 consensus_queue.cc:260] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.439339 7687 ts_tablet_manager.cc:1428] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 20:45:58.442559 7694 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.442975 7694 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.443744 7689 ts_tablet_manager.cc:1428] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260926 20:45:58.665668 7693 raft_consensus.cc:482] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:58.665875 7693 raft_consensus.cc:504] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.666591 7693 leader_election.cc:288] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:45:58.667094 7341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10626b5a8cb45078a3de53897c0de32" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f80aaefe62404eb18d0ea7f943a108" is_pre_election: true
I20260926 20:45:58.667155 7618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10626b5a8cb45078a3de53897c0de32" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b" is_pre_election: true
I20260926 20:45:58.667359 7618 raft_consensus.cc:2454] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 0.
I20260926 20:45:58.667382 7341 raft_consensus.cc:2454] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 0.
I20260926 20:45:58.667785 7410 leader_election.cc:302] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:45:58.668079 7693 raft_consensus.cc:2781] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:58.668181 7693 raft_consensus.cc:482] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:58.668274 7693 raft_consensus.cc:3037] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:58.670312 7693 raft_consensus.cc:504] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.670929 7693 leader_election.cc:288] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 election: Requested vote from peers b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:45:58.671411 7341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10626b5a8cb45078a3de53897c0de32" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f80aaefe62404eb18d0ea7f943a108"
I20260926 20:45:58.671468 7618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10626b5a8cb45078a3de53897c0de32" candidate_uuid: "5455b4187b0c4c92971999dcbf4a3246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b"
I20260926 20:45:58.671669 7341 raft_consensus.cc:3037] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:58.671695 7618 raft_consensus.cc:3037] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:58.674681 7341 raft_consensus.cc:2454] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 1.
I20260926 20:45:58.674681 7618 raft_consensus.cc:2454] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5455b4187b0c4c92971999dcbf4a3246 in term 1.
I20260926 20:45:58.675217 7410 leader_election.cc:302] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2776beeb947341478cf5bd36226ee61b, 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:45:58.675580 7693 raft_consensus.cc:2781] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:58.675789 7693 raft_consensus.cc:686] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 LEADER]: Becoming Leader. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Running, Role: LEADER
I20260926 20:45:58.676039 7693 consensus_queue.cc:237] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.677512 7698 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: TxnStatusTablet state changed. Reason: New leader 5455b4187b0c4c92971999dcbf4a3246. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.677835 7698 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:45:58.679126 7183 catalog_manager.cc:5495] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } health_report { overall_health: UNKNOWN } } }
I20260926 20:45:58.685261 7710 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 20:45:58.685564 7710 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 20:45:58.735801 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.728961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.5.78.2:43613)
0926 20:45:58.729491 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.729496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.729513 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:45:58.729591 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.729594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.729674 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.729804 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:58.730282 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.731300 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.733260 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.733703 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.733824 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.734479 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:58.735145 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:58.735255 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.735420 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.735492 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":279,"threads_started":1}
I20260926 20:45:58.735890 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.728902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.0.0.1:48378)
0926 20:45:58.729154 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.729177 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.729319 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.729852 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.729859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.729874 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:45:58.730096 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.730107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.731440 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.731447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.733084 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.733103 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.733897 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.733904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.734112 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.734191 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:45:58.735256 (+ 1065us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:45:58.735298 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.735398 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.735568 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 20:45:58.743538 7698 consensus_queue.cc:1035] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:58.745194 7341 raft_consensus.cc:1261] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Refusing update from remote peer 5455b4187b0c4c92971999dcbf4a3246: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:58.745297 7618 raft_consensus.cc:1261] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Refusing update from remote peer 5455b4187b0c4c92971999dcbf4a3246: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:58.745868 7715 consensus_queue.cc:1035] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:58.746253 7698 consensus_queue.cc:1035] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:58.769876 7695 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: TxnStatusTablet state changed. Reason: New leader 5455b4187b0c4c92971999dcbf4a3246. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.770181 7695 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.770977 7714 mvcc.cc:205] Tried to move back new op lower bound from 7333705968610082816 to 7333705968339038208. Current Snapshot: MvccSnapshot[applied={T|T < 7333705968610082816}]
I20260926 20:45:58.772675 7694 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: TxnStatusTablet state changed. Reason: New leader 5455b4187b0c4c92971999dcbf4a3246. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.772974 7694 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.771693 7715 consensus_queue.cc:1035] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260926 20:45:58.779240 7698 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.779497 7698 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:45:58.779834 7693 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.780155 7693 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 5455b4187b0c4c92971999dcbf4a3246: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:45:58.789968 7695 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.790256 7695 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P 2776beeb947341478cf5bd36226ee61b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.790199 7694 tablet_replica.cc:439] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5455b4187b0c4c92971999dcbf4a3246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } } }
I20260926 20:45:58.790436 7694 tablet_replica.cc:442] T a10626b5a8cb45078a3de53897c0de32 P b5f80aaefe62404eb18d0ea7f943a108: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:45:58.806502 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.799757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.5.78.2:43613)
0926 20:45:58.799978 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.799987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.800909 (+ 922us) server_negotiation.cc:407] Connection header received
0926 20:45:58.801123 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.801129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.801204 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.801308 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:58.801776 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.802636 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.804487 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.804986 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.805153 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.805626 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:58.805760 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:58.805869 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.806031 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.806122 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:45:58.825526 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.819191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.5.78.2:36613)
0926 20:45:58.819890 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.819910 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.820122 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.820503 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.820508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.820518 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:45:58.820918 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.820927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.821940 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.821948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.823333 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.823346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.824115 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.824123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.824247 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.824307 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:45:58.824854 (+ 547us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:45:58.824867 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.824956 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.825096 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":4,"thread_start_us":381,"threads_started":1}
I20260926 20:45:58.825555 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.819107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36613 (local address 127.5.78.2:43613)
0926 20:45:58.819287 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.819292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.820016 (+ 724us) server_negotiation.cc:407] Connection header received
0926 20:45:58.820285 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.820288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.820347 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.820497 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:58.821064 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.821820 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.823469 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.823902 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.824076 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.824576 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:58.824714 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:58.824816 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.825009 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.825162 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":51}
I20260926 20:45:58.895967 7181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:45:58.896279 7181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:45:58.900318 7181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Sending DeleteTablet for 3 replicas of tablet 106eb90e180743a09ecc287b613e3006
I20260926 20:45:58.901597 7598 tablet_service.cc:1514] Processing DeleteTablet for tablet 106eb90e180743a09ecc287b613e3006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:58 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40100
I20260926 20:45:58.901585 7320 tablet_service.cc:1514] Processing DeleteTablet for tablet 106eb90e180743a09ecc287b613e3006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:58 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:35716
I20260926 20:45:58.901611 7458 tablet_service.cc:1514] Processing DeleteTablet for tablet 106eb90e180743a09ecc287b613e3006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:45:58 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:48362
I20260926 20:45:58.907243 7181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:45:58.908144 7181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:45:58.913350 7733 tablet_replica.cc:330] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: stopping tablet replica
I20260926 20:45:58.914103 7734 tablet_replica.cc:330] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: stopping tablet replica
I20260926 20:45:58.914610 7734 raft_consensus.cc:2229] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:45:58.914880 7732 tablet_replica.cc:330] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: stopping tablet replica
I20260926 20:45:58.915360 7732 raft_consensus.cc:2229] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:45:58.915840 7732 raft_consensus.cc:2258] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:58.916695 7597 tablet_service.cc:1467] Processing CreateTablet for tablet d1e65b164c1e44d99d45d6a5786ef0ec (DEFAULT_TABLE table=test-table [id=7b9aa541aec4404bab1b54f07c18c750]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:58.916889 7733 raft_consensus.cc:2229] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:45:58.917270 7597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e65b164c1e44d99d45d6a5786ef0ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.917375 7459 tablet_service.cc:1467] Processing CreateTablet for tablet d1e65b164c1e44d99d45d6a5786ef0ec (DEFAULT_TABLE table=test-table [id=7b9aa541aec4404bab1b54f07c18c750]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:58.917809 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e65b164c1e44d99d45d6a5786ef0ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.917289 7733 raft_consensus.cc:2258] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:58.924623 7688 tablet_bootstrap.cc:492] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Bootstrap starting.
I20260926 20:45:58.922675 7320 tablet_service.cc:1467] Processing CreateTablet for tablet d1e65b164c1e44d99d45d6a5786ef0ec (DEFAULT_TABLE table=test-table [id=7b9aa541aec4404bab1b54f07c18c750]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:45:58.926241 7688 tablet_bootstrap.cc:654] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.926513 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e65b164c1e44d99d45d6a5786ef0ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:58.928193 7733 ts_tablet_manager.cc:1905] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:58.928810 7689 tablet_bootstrap.cc:492] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Bootstrap starting.
I20260926 20:45:58.931154 7689 tablet_bootstrap.cc:654] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.931375 7734 raft_consensus.cc:2258] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:45:58.934684 7734 ts_tablet_manager.cc:1905] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:58.939297 7733 ts_tablet_manager.cc:1918] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:45:58.939488 7733 log.cc:1196] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/106eb90e180743a09ecc287b613e3006
I20260926 20:45:58.942585 7732 ts_tablet_manager.cc:1905] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:45:58.943596 7688 tablet_bootstrap.cc:492] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: No bootstrap required, opened a new log
I20260926 20:45:58.943783 7688 ts_tablet_manager.cc:1397] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.000s
I20260926 20:45:58.944648 7688 raft_consensus.cc:348] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.944940 7688 raft_consensus.cc:374] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.945015 7688 raft_consensus.cc:729] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776beeb947341478cf5bd36226ee61b, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.945339 7688 consensus_queue.cc:260] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.945782 7689 tablet_bootstrap.cc:492] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: No bootstrap required, opened a new log
I20260926 20:45:58.946007 7689 ts_tablet_manager.cc:1397] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.001s
I20260926 20:45:58.947045 7689 raft_consensus.cc:348] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.947432 7689 raft_consensus.cc:374] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.947579 7689 raft_consensus.cc:729] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.947750 7733 ts_tablet_manager.cc:1939] T 106eb90e180743a09ecc287b613e3006 P 2776beeb947341478cf5bd36226ee61b: Deleting consensus metadata
I20260926 20:45:58.947997 7689 consensus_queue.cc:260] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.948828 7689 ts_tablet_manager.cc:1428] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:45:58.951457 7170 catalog_manager.cc:4841] TS 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821): tablet 106eb90e180743a09ecc287b613e3006 (table test-table [id=d577895002e34da2b04daa80dc7828d8]) successfully deleted
I20260926 20:45:58.952629 7737 tablet_bootstrap.cc:492] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Bootstrap starting.
I20260926 20:45:58.955075 7737 tablet_bootstrap.cc:654] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:58.955498 7688 ts_tablet_manager.cc:1428] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Time spent starting tablet: real 0.012s user 0.002s sys 0.001s
I20260926 20:45:58.956018 7732 ts_tablet_manager.cc:1918] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:45:58.957417 7732 log.cc:1196] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/106eb90e180743a09ecc287b613e3006
I20260926 20:45:58.957957 7732 ts_tablet_manager.cc:1939] T 106eb90e180743a09ecc287b613e3006 P b5f80aaefe62404eb18d0ea7f943a108: Deleting consensus metadata
I20260926 20:45:58.960081 7167 catalog_manager.cc:4841] TS b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711): tablet 106eb90e180743a09ecc287b613e3006 (table test-table [id=d577895002e34da2b04daa80dc7828d8]) successfully deleted
I20260926 20:45:58.965919 7734 ts_tablet_manager.cc:1918] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:45:58.966213 7734 log.cc:1196] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/106eb90e180743a09ecc287b613e3006
I20260926 20:45:58.966885 7734 ts_tablet_manager.cc:1939] T 106eb90e180743a09ecc287b613e3006 P 5455b4187b0c4c92971999dcbf4a3246: Deleting consensus metadata
I20260926 20:45:58.967972 7737 tablet_bootstrap.cc:492] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: No bootstrap required, opened a new log
I20260926 20:45:58.968165 7737 ts_tablet_manager.cc:1397] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.003s
I20260926 20:45:58.968921 7167 catalog_manager.cc:4841] TS 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613): tablet 106eb90e180743a09ecc287b613e3006 (table test-table [id=d577895002e34da2b04daa80dc7828d8]) successfully deleted
I20260926 20:45:58.969075 7737 raft_consensus.cc:348] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.969317 7737 raft_consensus.cc:374] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:58.969434 7737 raft_consensus.cc:729] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Initialized, Role: FOLLOWER
I20260926 20:45:58.969697 7737 consensus_queue.cc:260] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.970330 7737 ts_tablet_manager.cc:1428] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 20:45:58.980122 7694 raft_consensus.cc:482] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:45:58.980381 7694 raft_consensus.cc:504] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.982638 7694 leader_election.cc:288] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:45:58.991444 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.982971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:38615 (local address 127.5.78.2:43613)
0926 20:45:58.983437 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.983445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.984124 (+ 679us) server_negotiation.cc:407] Connection header received
0926 20:45:58.984445 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.984453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.984543 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.984619 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.985515 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.986388 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.988647 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.990095 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.990226 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.990347 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.990958 (+ 611us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.991121 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260926 20:45:58.991544 7739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.982991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.5.78.1:38615)
0926 20:45:58.983993 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.984015 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.984259 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.984861 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.984865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.984891 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.985339 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.985361 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.986573 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.986581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.988449 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.988467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.990561 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.990568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.990756 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.990789 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.990896 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.991052 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":63,"spinlock_wait_cycles":13056,"thread_start_us":782,"threads_started":1}
I20260926 20:45:58.992775 7479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e65b164c1e44d99d45d6a5786ef0ec" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5455b4187b0c4c92971999dcbf4a3246" is_pre_election: true
I20260926 20:45:58.993162 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.983610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:55501 (local address 127.5.78.3:36821)
0926 20:45:58.984798 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:58.984805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:58.984829 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:45:58.984891 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:58.984896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:58.984985 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:58.985091 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:45:58.985922 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.987236 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.990100 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.991525 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.992138 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.992247 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:58.992663 (+ 416us) server_negotiation.cc:299] Negotiation successful
0926 20:45:58.992845 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":882,"threads_started":1}
I20260926 20:45:58.993263 7740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:58.983159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:36821 (local address 127.5.78.1:55501)
0926 20:45:58.984084 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:58.984100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:58.984259 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:58.985148 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:58.985154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:58.985188 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:45:58.985672 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.985685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.987681 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.987686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.989869 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:58.989890 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:58.992195 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:58.992201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:58.992459 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:58.992497 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:45:58.992607 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 20:45:58.992775 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":3,"spinlock_wait_cycles":31744,"thread_start_us":929,"threads_started":1}
I20260926 20:45:58.993273 7479 raft_consensus.cc:2454] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 0.
I20260926 20:45:58.993949 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e65b164c1e44d99d45d6a5786ef0ec" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b" is_pre_election: true
I20260926 20:45:58.994211 7617 raft_consensus.cc:2454] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 0.
I20260926 20:45:58.994300 7270 leader_election.cc:302] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:45:58.994868 7694 raft_consensus.cc:2781] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:45:58.995002 7694 raft_consensus.cc:482] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:45:58.995091 7694 raft_consensus.cc:3037] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:58.998457 7694 raft_consensus.cc:504] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:58.999369 7694 leader_election.cc:288] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 election: Requested vote from peers 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:45:59.000069 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e65b164c1e44d99d45d6a5786ef0ec" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b"
I20260926 20:45:59.000308 7617 raft_consensus.cc:3037] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:59.002764 7479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e65b164c1e44d99d45d6a5786ef0ec" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5455b4187b0c4c92971999dcbf4a3246"
I20260926 20:45:59.003031 7479 raft_consensus.cc:3037] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:59.004190 7617 raft_consensus.cc:2454] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 1.
I20260926 20:45:59.004733 7273 leader_election.cc:302] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2776beeb947341478cf5bd36226ee61b, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:45:59.005131 7694 raft_consensus.cc:2781] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:59.005622 7694 raft_consensus.cc:686] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 1 LEADER]: Becoming Leader. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Running, Role: LEADER
I20260926 20:45:59.006072 7694 consensus_queue.cc:237] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:45:59.006539 7479 raft_consensus.cc:2454] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 1.
I20260926 20:45:59.010324 7183 catalog_manager.cc:5495] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1). New cstate: current_term: 1 leader_uuid: "b5f80aaefe62404eb18d0ea7f943a108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } health_report { overall_health: UNKNOWN } } }
I20260926 20:45:59.045363 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:59.038727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.5.78.1:43711)
0926 20:45:59.039596 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:59.039600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:59.039619 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:45:59.039701 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:59.039704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:59.039787 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:59.039967 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:59.040410 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.041189 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.042541 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.043214 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.043360 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:59.043787 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:59.044623 (+ 836us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:59.044786 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:59.044888 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 20:45:59.044979 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":625,"threads_started":1}
I20260926 20:45:59.063769 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:59.056688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.5.78.1:60339)
0926 20:45:59.057403 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:59.057421 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:59.057588 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:59.057958 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:59.057961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:59.057971 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:45:59.058251 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:59.058259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.059766 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.059772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:59.060976 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:59.060990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.062061 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.062066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:59.062324 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:59.062458 (+ 134us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:45:59.063059 (+ 601us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:45:59.063073 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 20:45:59.063189 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 20:45:59.063349 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":372,"threads_started":1}
I20260926 20:45:59.063822 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:59.056624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:60339 (local address 127.5.78.2:43613)
0926 20:45:59.056784 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:59.056788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:59.057547 (+ 759us) server_negotiation.cc:407] Connection header received
0926 20:45:59.057723 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:59.057729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:59.057820 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:59.057949 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:59.058394 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.059603 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.061141 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.061837 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.061990 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:59.062687 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:59.062932 (+ 245us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:59.063033 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:59.063231 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 20:45:59.063370 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 20:45:59.077368 7617 raft_consensus.cc:1261] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Refusing update from remote peer b5f80aaefe62404eb18d0ea7f943a108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:59.077509 7479 raft_consensus.cc:1261] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Refusing update from remote peer b5f80aaefe62404eb18d0ea7f943a108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:59.078199 7724 consensus_queue.cc:1035] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:59.079037 7694 consensus_queue.cc:1035] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:59.139356 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:59.120094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43711 (local address 127.5.78.2:57055)
0926 20:45:59.120457 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:59.120476 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:59.120597 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:59.121887 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:59.121893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:59.121907 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:45:59.122101 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:59.122112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.124481 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.124488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:59.126340 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:59.126361 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.135758 (+ 9397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.135765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:59.135986 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:59.136050 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:45:59.138674 (+ 2624us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:45:59.138694 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 20:45:59.138788 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 20:45:59.138967 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260926 20:45:59.140429 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:59.121263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:57055 (local address 127.5.78.1:43711)
0926 20:45:59.121438 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:59.121445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:59.121465 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:45:59.121530 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:59.121536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:59.121614 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:59.121716 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:59.122286 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.123448 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.131156 (+ 7708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:59.131845 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:59.131978 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:59.136247 (+ 4269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:59.136387 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:59.136464 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:59.140049 (+ 3585us) server_negotiation.cc:299] Negotiation successful
0926 20:45:59.140150 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 20:45:59.141589 7718 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 20:46:00.775915 7181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:00.776139 7181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:00.780618 7181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Sending DeleteTablet for 3 replicas of tablet d1e65b164c1e44d99d45d6a5786ef0ec
I20260926 20:46:00.781679 7320 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e65b164c1e44d99d45d6a5786ef0ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:00 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:35716
I20260926 20:46:00.781677 7597 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e65b164c1e44d99d45d6a5786ef0ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:00 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40100
I20260926 20:46:00.781703 7459 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e65b164c1e44d99d45d6a5786ef0ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:00 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:48362
I20260926 20:46:00.793525 7775 tablet_replica.cc:330] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: stopping tablet replica
I20260926 20:46:00.793799 7775 raft_consensus.cc:2229] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:00.794090 7775 raft_consensus.cc:2258] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:00.794126 7774 tablet_replica.cc:330] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: stopping tablet replica
I20260926 20:46:00.794301 7774 raft_consensus.cc:2229] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:00.794960 7774 raft_consensus.cc:2258] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:00.795310 7775 ts_tablet_manager.cc:1905] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:00.796043 7774 ts_tablet_manager.cc:1905] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:00.796406 7773 tablet_replica.cc:330] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: stopping tablet replica
I20260926 20:46:00.796686 7773 raft_consensus.cc:2229] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:00.796989 7773 raft_consensus.cc:2258] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:00.798226 7773 ts_tablet_manager.cc:1905] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:00.802444 7775 ts_tablet_manager.cc:1918] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:46:00.802603 7775 log.cc:1196] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/d1e65b164c1e44d99d45d6a5786ef0ec
I20260926 20:46:00.802645 7774 ts_tablet_manager.cc:1918] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:46:00.802779 7774 log.cc:1196] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/d1e65b164c1e44d99d45d6a5786ef0ec
I20260926 20:46:00.803103 7775 ts_tablet_manager.cc:1939] T d1e65b164c1e44d99d45d6a5786ef0ec P 2776beeb947341478cf5bd36226ee61b: Deleting consensus metadata
I20260926 20:46:00.803243 7774 ts_tablet_manager.cc:1939] T d1e65b164c1e44d99d45d6a5786ef0ec P b5f80aaefe62404eb18d0ea7f943a108: Deleting consensus metadata
I20260926 20:46:00.804283 7170 catalog_manager.cc:4841] TS 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821): tablet d1e65b164c1e44d99d45d6a5786ef0ec (table test-table [id=7b9aa541aec4404bab1b54f07c18c750]) successfully deleted
I20260926 20:46:00.804283 7167 catalog_manager.cc:4841] TS b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711): tablet d1e65b164c1e44d99d45d6a5786ef0ec (table test-table [id=7b9aa541aec4404bab1b54f07c18c750]) successfully deleted
I20260926 20:46:00.805794 7773 ts_tablet_manager.cc:1918] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:46:00.805917 7773 log.cc:1196] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/d1e65b164c1e44d99d45d6a5786ef0ec
I20260926 20:46:00.806327 7773 ts_tablet_manager.cc:1939] T d1e65b164c1e44d99d45d6a5786ef0ec P 5455b4187b0c4c92971999dcbf4a3246: Deleting consensus metadata
I20260926 20:46:00.807399 7167 catalog_manager.cc:4841] TS 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613): tablet d1e65b164c1e44d99d45d6a5786ef0ec (table test-table [id=7b9aa541aec4404bab1b54f07c18c750]) successfully deleted
Sep 26 20:46:00 dist-test-slave-z7j8 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:46:00 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:00.841284 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:00.826691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.5.78.62:44623)
0926 20:46:00.827366 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:00.827373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:00.827483 (+ 110us) server_negotiation.cc:407] Connection header received
0926 20:46:00.827708 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:00.827712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:00.827773 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:00.827903 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:00.829190 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:00.829950 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:00.831318 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:00.831707 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:00.831860 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:00.836092 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:00.836098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:00.836101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:00.836135 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:00.838770 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:00.839427 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:00.839432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:00.839434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:00.839545 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:00.839889 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:00.839893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:00.839896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:00.840163 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:00.840412 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:00.840705 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 20:46:00.840938 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":329,"threads_started":1}
I20260926 20:46:00.844966 7181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:00.845652 7181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:00.852547 7320 tablet_service.cc:1467] Processing CreateTablet for tablet 3333920f014b49e38218807c0ee0cfac (DEFAULT_TABLE table=test-table [id=86f722e60acc4879a9b70ed350562a1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:00.852612 7459 tablet_service.cc:1467] Processing CreateTablet for tablet 3333920f014b49e38218807c0ee0cfac (DEFAULT_TABLE table=test-table [id=86f722e60acc4879a9b70ed350562a1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:00.852837 7597 tablet_service.cc:1467] Processing CreateTablet for tablet 3333920f014b49e38218807c0ee0cfac (DEFAULT_TABLE table=test-table [id=86f722e60acc4879a9b70ed350562a1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:00.853021 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3333920f014b49e38218807c0ee0cfac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:00.853060 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3333920f014b49e38218807c0ee0cfac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:00.853230 7597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3333920f014b49e38218807c0ee0cfac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:00.860075 7784 tablet_bootstrap.cc:492] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Bootstrap starting.
I20260926 20:46:00.860814 7785 tablet_bootstrap.cc:492] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Bootstrap starting.
I20260926 20:46:00.862213 7784 tablet_bootstrap.cc:654] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:00.862634 7785 tablet_bootstrap.cc:654] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:00.864091 7786 tablet_bootstrap.cc:492] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Bootstrap starting.
I20260926 20:46:00.866384 7786 tablet_bootstrap.cc:654] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:00.867942 7785 tablet_bootstrap.cc:492] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: No bootstrap required, opened a new log
I20260926 20:46:00.868119 7785 ts_tablet_manager.cc:1397] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260926 20:46:00.869141 7784 tablet_bootstrap.cc:492] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: No bootstrap required, opened a new log
I20260926 20:46:00.869095 7785 raft_consensus.cc:348] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.869324 7784 ts_tablet_manager.cc:1397] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260926 20:46:00.869354 7785 raft_consensus.cc:374] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:00.869439 7786 tablet_bootstrap.cc:492] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: No bootstrap required, opened a new log
I20260926 20:46:00.869463 7785 raft_consensus.cc:729] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776beeb947341478cf5bd36226ee61b, State: Initialized, Role: FOLLOWER
I20260926 20:46:00.869612 7786 ts_tablet_manager.cc:1397] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 20:46:00.869709 7785 consensus_queue.cc:260] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.870205 7784 raft_consensus.cc:348] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.870498 7784 raft_consensus.cc:374] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:00.870599 7784 raft_consensus.cc:729] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Initialized, Role: FOLLOWER
I20260926 20:46:00.870628 7785 ts_tablet_manager.cc:1428] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 20:46:00.870514 7786 raft_consensus.cc:348] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.870755 7786 raft_consensus.cc:374] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:00.870859 7786 raft_consensus.cc:729] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Initialized, Role: FOLLOWER
I20260926 20:46:00.870807 7784 consensus_queue.cc:260] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.871104 7786 consensus_queue.cc:260] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.871366 7784 ts_tablet_manager.cc:1428] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:46:00.871712 7786 ts_tablet_manager.cc:1428] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 20:46:00.897792 7768 raft_consensus.cc:482] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:00.898046 7768 raft_consensus.cc:504] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.898766 7768 leader_election.cc:288] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:46:00.899300 7618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3333920f014b49e38218807c0ee0cfac" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b" is_pre_election: true
I20260926 20:46:00.899290 7478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3333920f014b49e38218807c0ee0cfac" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5455b4187b0c4c92971999dcbf4a3246" is_pre_election: true
I20260926 20:46:00.899552 7478 raft_consensus.cc:2454] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 0.
I20260926 20:46:00.899555 7618 raft_consensus.cc:2454] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 0.
I20260926 20:46:00.900053 7273 leader_election.cc:302] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2776beeb947341478cf5bd36226ee61b, 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:46:00.900336 7768 raft_consensus.cc:2781] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:00.900453 7768 raft_consensus.cc:482] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:00.900542 7768 raft_consensus.cc:3037] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:00.902698 7768 raft_consensus.cc:504] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.903275 7768 leader_election.cc:288] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 election: Requested vote from peers 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613), 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821)
I20260926 20:46:00.903853 7618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3333920f014b49e38218807c0ee0cfac" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2776beeb947341478cf5bd36226ee61b"
I20260926 20:46:00.903892 7478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3333920f014b49e38218807c0ee0cfac" candidate_uuid: "b5f80aaefe62404eb18d0ea7f943a108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5455b4187b0c4c92971999dcbf4a3246"
I20260926 20:46:00.904099 7478 raft_consensus.cc:3037] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:00.904124 7618 raft_consensus.cc:3037] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:00.906497 7478 raft_consensus.cc:2454] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 1.
I20260926 20:46:00.906561 7618 raft_consensus.cc:2454] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f80aaefe62404eb18d0ea7f943a108 in term 1.
I20260926 20:46:00.906967 7270 leader_election.cc:302] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5455b4187b0c4c92971999dcbf4a3246, b5f80aaefe62404eb18d0ea7f943a108; no voters:
I20260926 20:46:00.907258 7768 raft_consensus.cc:2781] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:00.907416 7768 raft_consensus.cc:686] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 1 LEADER]: Becoming Leader. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Running, Role: LEADER
I20260926 20:46:00.907709 7768 consensus_queue.cc:237] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:00.910283 7180 catalog_manager.cc:5495] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1). New cstate: current_term: 1 leader_uuid: "b5f80aaefe62404eb18d0ea7f943a108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } health_report { overall_health: UNKNOWN } } }
I20260926 20:46:00.938788 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:00.933631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.5.78.1:43711)
0926 20:46:00.934257 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:00.934261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:00.934278 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:46:00.934333 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:00.934337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:00.934398 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:00.934552 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:00.934981 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:00.935728 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:00.937000 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:00.937461 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:00.937593 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:00.938003 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:00.938114 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:00.938208 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:00.938356 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 20:46:00.938464 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":357,"threads_started":1}
I20260926 20:46:00.948159 7618 raft_consensus.cc:1261] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Refusing update from remote peer b5f80aaefe62404eb18d0ea7f943a108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:00.948184 7478 raft_consensus.cc:1261] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Refusing update from remote peer b5f80aaefe62404eb18d0ea7f943a108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:00.948728 7768 consensus_queue.cc:1035] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:00.949059 7764 consensus_queue.cc:1035] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:00.954687 7756 mvcc.cc:205] Tried to move back new op lower bound from 7333705977638240256 to 7333705977480138752. Current Snapshot: MvccSnapshot[applied={T|T < 7333705977638240256}]
I20260926 20:46:00.995093 7180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:00.995348 7180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:00.998338 7180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Sending DeleteTablet for 3 replicas of tablet 3333920f014b49e38218807c0ee0cfac
I20260926 20:46:00.999328 7597 tablet_service.cc:1514] Processing DeleteTablet for tablet 3333920f014b49e38218807c0ee0cfac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:00 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40100
I20260926 20:46:00.999414 7320 tablet_service.cc:1514] Processing DeleteTablet for tablet 3333920f014b49e38218807c0ee0cfac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:00 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:35716
I20260926 20:46:00.999639 7775 tablet_replica.cc:330] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: stopping tablet replica
I20260926 20:46:00.999670 7774 tablet_replica.cc:330] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: stopping tablet replica
I20260926 20:46:00.999850 7775 raft_consensus.cc:2229] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:00.999845 7774 raft_consensus.cc:2229] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:01.000046 7775 raft_consensus.cc:2258] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:01.000170 7774 raft_consensus.cc:2258] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:01.000505 7459 tablet_service.cc:1514] Processing DeleteTablet for tablet 3333920f014b49e38218807c0ee0cfac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:00 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:48362
I20260926 20:46:01.000429 7180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 20:46:01.000808 7773 tablet_replica.cc:330] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: stopping tablet replica
I20260926 20:46:01.001004 7773 raft_consensus.cc:2229] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 20:46:01.001065 7180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:01.001190 7773 raft_consensus.cc:2258] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:01.001360 7775 ts_tablet_manager.cc:1905] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:01.001360 7774 ts_tablet_manager.cc:1905] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:01.002449 7773 ts_tablet_manager.cc:1905] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:01.007555 7773 ts_tablet_manager.cc:1918] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:01.007699 7773 log.cc:1196] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/3333920f014b49e38218807c0ee0cfac
I20260926 20:46:01.008213 7774 ts_tablet_manager.cc:1918] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:01.008213 7775 ts_tablet_manager.cc:1918] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:01.008386 7775 log.cc:1196] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/3333920f014b49e38218807c0ee0cfac
I20260926 20:46:01.008386 7774 log.cc:1196] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/3333920f014b49e38218807c0ee0cfac
I20260926 20:46:01.008441 7773 ts_tablet_manager.cc:1939] T 3333920f014b49e38218807c0ee0cfac P 5455b4187b0c4c92971999dcbf4a3246: Deleting consensus metadata
I20260926 20:46:01.008778 7775 ts_tablet_manager.cc:1939] T 3333920f014b49e38218807c0ee0cfac P 2776beeb947341478cf5bd36226ee61b: Deleting consensus metadata
I20260926 20:46:01.008798 7774 ts_tablet_manager.cc:1939] T 3333920f014b49e38218807c0ee0cfac P b5f80aaefe62404eb18d0ea7f943a108: Deleting consensus metadata
I20260926 20:46:01.009860 7167 catalog_manager.cc:4841] TS 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613): tablet 3333920f014b49e38218807c0ee0cfac (table test-table [id=86f722e60acc4879a9b70ed350562a1b]) successfully deleted
I20260926 20:46:01.009966 7170 catalog_manager.cc:4841] TS 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821): tablet 3333920f014b49e38218807c0ee0cfac (table test-table [id=86f722e60acc4879a9b70ed350562a1b]) successfully deleted
I20260926 20:46:01.011051 7167 catalog_manager.cc:4841] TS b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711): tablet 3333920f014b49e38218807c0ee0cfac (table test-table [id=86f722e60acc4879a9b70ed350562a1b]) successfully deleted
I20260926 20:46:01.014060 7320 tablet_service.cc:1467] Processing CreateTablet for tablet 626902d62d734ab3bd27e0aa18183746 (DEFAULT_TABLE table=test-table [id=34cec9f33d174715b242db1e58f8dbbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:01.014295 7459 tablet_service.cc:1467] Processing CreateTablet for tablet 626902d62d734ab3bd27e0aa18183746 (DEFAULT_TABLE table=test-table [id=34cec9f33d174715b242db1e58f8dbbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:01.014569 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626902d62d734ab3bd27e0aa18183746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:01.014528 7597 tablet_service.cc:1467] Processing CreateTablet for tablet 626902d62d734ab3bd27e0aa18183746 (DEFAULT_TABLE table=test-table [id=34cec9f33d174715b242db1e58f8dbbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:01.014714 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626902d62d734ab3bd27e0aa18183746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:01.014915 7597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626902d62d734ab3bd27e0aa18183746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:01.019448 7786 tablet_bootstrap.cc:492] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Bootstrap starting.
I20260926 20:46:01.020800 7786 tablet_bootstrap.cc:654] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:01.021159 7784 tablet_bootstrap.cc:492] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Bootstrap starting.
I20260926 20:46:01.021723 7785 tablet_bootstrap.cc:492] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Bootstrap starting.
I20260926 20:46:01.022708 7784 tablet_bootstrap.cc:654] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:01.023003 7786 tablet_bootstrap.cc:492] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: No bootstrap required, opened a new log
I20260926 20:46:01.023156 7786 ts_tablet_manager.cc:1397] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260926 20:46:01.023346 7785 tablet_bootstrap.cc:654] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:01.024011 7786 raft_consensus.cc:348] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.024339 7786 raft_consensus.cc:374] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:01.024428 7786 raft_consensus.cc:729] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5455b4187b0c4c92971999dcbf4a3246, State: Initialized, Role: FOLLOWER
I20260926 20:46:01.024756 7786 consensus_queue.cc:260] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.025854 7785 tablet_bootstrap.cc:492] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: No bootstrap required, opened a new log
I20260926 20:46:01.025617 7786 ts_tablet_manager.cc:1428] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 20:46:01.026155 7785 ts_tablet_manager.cc:1397] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 20:46:01.026965 7785 raft_consensus.cc:348] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.027231 7785 raft_consensus.cc:374] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:01.027311 7785 raft_consensus.cc:729] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776beeb947341478cf5bd36226ee61b, State: Initialized, Role: FOLLOWER
I20260926 20:46:01.027571 7785 consensus_queue.cc:260] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.028128 7785 ts_tablet_manager.cc:1428] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 20:46:01.031853 7784 tablet_bootstrap.cc:492] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: No bootstrap required, opened a new log
I20260926 20:46:01.032066 7784 ts_tablet_manager.cc:1397] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260926 20:46:01.032685 7784 raft_consensus.cc:348] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.032886 7784 raft_consensus.cc:374] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:01.032965 7784 raft_consensus.cc:729] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f80aaefe62404eb18d0ea7f943a108, State: Initialized, Role: FOLLOWER
I20260926 20:46:01.033200 7784 consensus_queue.cc:260] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.033767 7784 ts_tablet_manager.cc:1428] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 20:46:01.175442 7790 raft_consensus.cc:482] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:01.175732 7790 raft_consensus.cc:504] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.177466 7790 leader_election.cc:288] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711), 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613)
I20260926 20:46:01.184762 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.177761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:37667 (local address 127.5.78.1:43711)
0926 20:46:01.177985 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:01.177992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:01.178371 (+ 379us) server_negotiation.cc:407] Connection header received
0926 20:46:01.178686 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:01.178694 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:01.178789 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:01.178893 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:01.179650 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.180535 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.182728 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.183601 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.183763 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.183896 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:01.184294 (+ 398us) server_negotiation.cc:299] Negotiation successful
0926 20:46:01.184437 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:46:01.184775 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.178541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:40973 (local address 127.5.78.2:43613)
0926 20:46:01.179391 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:01.179395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:01.179416 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:46:01.179471 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:01.179474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:01.179530 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:01.179650 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:01.180203 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.180942 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.182793 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.183601 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.183752 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.183904 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:01.184275 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 20:46:01.184418 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":597,"threads_started":1}
I20260926 20:46:01.184962 7802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.178362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.5.78.3:40973)
0926 20:46:01.179139 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:01.179166 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:01.179285 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:01.179664 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:01.179668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:01.179693 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:01.180015 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.180023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.181078 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.181085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.182626 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.182639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.183838 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.183844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.184087 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.184129 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:46:01.184234 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:46:01.184416 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":37,"thread_start_us":427,"threads_started":1}
I20260926 20:46:01.185348 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.177645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43711 (local address 127.5.78.3:37667)
0926 20:46:01.178229 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:01.178274 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:01.178526 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:01.179077 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:01.179082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:01.179107 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:01.179482 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.179492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.180681 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.180687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.182554 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.182569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.183884 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.183891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.184126 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.184152 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 20:46:01.184275 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 20:46:01.184436 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":122,"thread_start_us":275,"threads_started":1}
I20260926 20:46:01.185685 7478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626902d62d734ab3bd27e0aa18183746" candidate_uuid: "2776beeb947341478cf5bd36226ee61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5455b4187b0c4c92971999dcbf4a3246" is_pre_election: true
I20260926 20:46:01.185988 7478 raft_consensus.cc:2454] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2776beeb947341478cf5bd36226ee61b in term 0.
I20260926 20:46:01.186064 7339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626902d62d734ab3bd27e0aa18183746" candidate_uuid: "2776beeb947341478cf5bd36226ee61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f80aaefe62404eb18d0ea7f943a108" is_pre_election: true
I20260926 20:46:01.186358 7339 raft_consensus.cc:2454] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2776beeb947341478cf5bd36226ee61b in term 0.
I20260926 20:46:01.186602 7549 leader_election.cc:302] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2776beeb947341478cf5bd36226ee61b, 5455b4187b0c4c92971999dcbf4a3246; no voters:
I20260926 20:46:01.187150 7790 raft_consensus.cc:2781] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:01.187306 7790 raft_consensus.cc:482] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:01.187427 7790 raft_consensus.cc:3037] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:01.189605 7790 raft_consensus.cc:504] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.190347 7790 leader_election.cc:288] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [CANDIDATE]: Term 1 election: Requested vote from peers b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711), 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613)
I20260926 20:46:01.190956 7339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626902d62d734ab3bd27e0aa18183746" candidate_uuid: "2776beeb947341478cf5bd36226ee61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f80aaefe62404eb18d0ea7f943a108"
I20260926 20:46:01.191041 7478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626902d62d734ab3bd27e0aa18183746" candidate_uuid: "2776beeb947341478cf5bd36226ee61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5455b4187b0c4c92971999dcbf4a3246"
I20260926 20:46:01.191185 7339 raft_consensus.cc:3037] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:01.191296 7478 raft_consensus.cc:3037] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:01.193558 7478 raft_consensus.cc:2454] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2776beeb947341478cf5bd36226ee61b in term 1.
I20260926 20:46:01.193583 7339 raft_consensus.cc:2454] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2776beeb947341478cf5bd36226ee61b in term 1.
I20260926 20:46:01.194111 7549 leader_election.cc:302] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2776beeb947341478cf5bd36226ee61b, 5455b4187b0c4c92971999dcbf4a3246; no voters:
I20260926 20:46:01.194504 7790 raft_consensus.cc:2781] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:01.195032 7790 raft_consensus.cc:686] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 1 LEADER]: Becoming Leader. State: Replica: 2776beeb947341478cf5bd36226ee61b, State: Running, Role: LEADER
I20260926 20:46:01.195425 7790 consensus_queue.cc:237] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } }
I20260926 20:46:01.199187 7180 catalog_manager.cc:5495] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2776beeb947341478cf5bd36226ee61b (127.5.78.3). New cstate: current_term: 1 leader_uuid: "2776beeb947341478cf5bd36226ee61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2776beeb947341478cf5bd36226ee61b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36821 } health_report { overall_health: HEALTHY } } }
I20260926 20:46:01.234364 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.228000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.5.78.3:36821)
0926 20:46:01.228575 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:01.228579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:01.228601 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:46:01.228688 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:01.228692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:01.228767 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:01.228933 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:01.229393 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.230179 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.231741 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.232211 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.232360 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.232904 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:01.233583 (+ 679us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:01.233708 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:01.233862 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 20:46:01.233938 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":325,"threads_started":1}
I20260926 20:46:01.251811 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.246064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:60795 (local address 127.5.78.2:43613)
0926 20:46:01.246360 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:01.246366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:01.246743 (+ 377us) server_negotiation.cc:407] Connection header received
0926 20:46:01.246920 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:01.246924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:01.246992 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:01.247075 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:01.247615 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.248308 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.249834 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.250260 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.250460 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.251139 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:01.251283 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:01.251364 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:01.251522 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 20:46:01.251590 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260926 20:46:01.252094 7809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.245889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:43613 (local address 127.5.78.3:60795)
0926 20:46:01.246635 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:01.246654 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:01.246803 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:01.247145 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:01.247149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:01.247162 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:01.247473 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.247480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.248451 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.248456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.249679 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.249692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.250690 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.250696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.250861 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.250931 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:01.251396 (+ 465us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:01.251408 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:46:01.251493 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 20:46:01.251644 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":354,"threads_started":1}
I20260926 20:46:01.260859 7478 raft_consensus.cc:1261] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Refusing update from remote peer 2776beeb947341478cf5bd36226ee61b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:01.260951 7339 raft_consensus.cc:1261] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Refusing update from remote peer 2776beeb947341478cf5bd36226ee61b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:01.261538 7790 consensus_queue.cc:1035] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5455b4187b0c4c92971999dcbf4a3246" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:01.261895 7804 consensus_queue.cc:1035] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f80aaefe62404eb18d0ea7f943a108" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:01.296777 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.284196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:53727 (local address 127.5.78.3:36821)
0926 20:46:01.284406 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:01.284411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:01.291568 (+ 7157us) server_negotiation.cc:407] Connection header received
0926 20:46:01.291727 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:01.291732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:01.291844 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:01.291986 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:01.292644 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.293669 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.295138 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.295562 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.295673 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.296075 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:01.296180 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:01.296252 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:01.296424 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 20:46:01.296492 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260926 20:46:01.296844 7814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:01.284092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:36821 (local address 127.5.78.2:53727)
0926 20:46:01.291443 (+ 7351us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:01.291460 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:01.291588 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:01.292031 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:01.292036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:01.292049 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:01.292399 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.292411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.293824 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.293828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.294987 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:01.295000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:01.295738 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:01.295742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:01.295884 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:01.295934 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:01.296299 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:01.296310 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:46:01.296392 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 20:46:01.296563 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7197,"thread_start_us":7027,"threads_started":1}
W20260926 20:46:01.298062 7755 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 20:46:02.894006 7180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:02.894238 7180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:02.898608 7180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5505554f10cc4a9fad28f3342ebebc35: Sending DeleteTablet for 3 replicas of tablet 626902d62d734ab3bd27e0aa18183746
I20260926 20:46:02.899461 7320 tablet_service.cc:1514] Processing DeleteTablet for tablet 626902d62d734ab3bd27e0aa18183746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:02 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:35716
I20260926 20:46:02.899538 7597 tablet_service.cc:1514] Processing DeleteTablet for tablet 626902d62d734ab3bd27e0aa18183746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:02 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40100
I20260926 20:46:02.899571 7458 tablet_service.cc:1514] Processing DeleteTablet for tablet 626902d62d734ab3bd27e0aa18183746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:02 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:48362
I20260926 20:46:02.900138 7840 tablet_replica.cc:330] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: stopping tablet replica
I20260926 20:46:02.900401 7840 raft_consensus.cc:2229] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:02.900588 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7254
I20260926 20:46:02.901124 7840 raft_consensus.cc:2258] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:02.902607 7840 ts_tablet_manager.cc:1905] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:02.911130 7841 tablet_replica.cc:330] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: stopping tablet replica
I20260926 20:46:02.911404 7841 raft_consensus.cc:2229] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:02.911677 7841 raft_consensus.cc:2258] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:02.912101 7840 ts_tablet_manager.cc:1918] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:46:02.912317 7840 log.cc:1196] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/626902d62d734ab3bd27e0aa18183746
I20260926 20:46:02.912899 7840 ts_tablet_manager.cc:1939] T 626902d62d734ab3bd27e0aa18183746 P 2776beeb947341478cf5bd36226ee61b: Deleting consensus metadata
I20260926 20:46:02.913280 7841 ts_tablet_manager.cc:1905] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:02.914297 7170 catalog_manager.cc:4841] TS 2776beeb947341478cf5bd36226ee61b (127.5.78.3:36821): tablet 626902d62d734ab3bd27e0aa18183746 (table test-table [id=34cec9f33d174715b242db1e58f8dbbc]) successfully deleted
I20260926 20:46:02.916359 7839 tablet_replica.cc:330] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: stopping tablet replica
I20260926 20:46:02.916560 7839 raft_consensus.cc:2229] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:02.916802 7839 raft_consensus.cc:2258] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:02.917651 7839 ts_tablet_manager.cc:1905] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:02.919011 7841 ts_tablet_manager.cc:1918] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:46:02.919159 7841 log.cc:1196] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/626902d62d734ab3bd27e0aa18183746
I20260926 20:46:02.919576 7841 ts_tablet_manager.cc:1939] T 626902d62d734ab3bd27e0aa18183746 P 5455b4187b0c4c92971999dcbf4a3246: Deleting consensus metadata
I20260926 20:46:02.920550 7167 catalog_manager.cc:4841] TS 5455b4187b0c4c92971999dcbf4a3246 (127.5.78.2:43613): tablet 626902d62d734ab3bd27e0aa18183746 (table test-table [id=34cec9f33d174715b242db1e58f8dbbc]) successfully deleted
I20260926 20:46:02.922578 7839 ts_tablet_manager.cc:1918] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:46:02.922735 7839 log.cc:1196] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/626902d62d734ab3bd27e0aa18183746
I20260926 20:46:02.923228 7839 ts_tablet_manager.cc:1939] T 626902d62d734ab3bd27e0aa18183746 P b5f80aaefe62404eb18d0ea7f943a108: Deleting consensus metadata
I20260926 20:46:02.924304 7167 catalog_manager.cc:4841] TS b5f80aaefe62404eb18d0ea7f943a108 (127.5.78.1:43711): tablet 626902d62d734ab3bd27e0aa18183746 (table test-table [id=34cec9f33d174715b242db1e58f8dbbc]) successfully deleted
Sep 26 20:46:02 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:02.932214 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:02.903609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.5.78.1:43711)
0926 20:46:02.910922 (+ 7313us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:02.910928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:02.910955 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:46:02.911022 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:02.911026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:02.911107 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:02.911239 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:02.913770 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:02.915485 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:02.917479 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:02.918137 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:02.918272 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:02.926521 (+ 8249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:02.926547 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:02.926564 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:02.926604 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:02.929499 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:02.930092 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:02.930099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:02.930102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:02.930230 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:02.930713 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:02.930722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:02.930725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:02.931078 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:02.931345 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:02.931640 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 20:46:02.931872 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7159,"thread_start_us":7004,"threads_started":1}
W20260926 20:46:02.932807 7385 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35732
W20260926 20:46:02.933388 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7254: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:46:02.933583 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 7254
I20260926 20:46:02.955986 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7394
Sep 26 20:46:02 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:46:02.972249 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:02.956806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.5.78.2:43613)
0926 20:46:02.957380 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:02.957384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:02.957420 (+ 36us) server_negotiation.cc:407] Connection header received
0926 20:46:02.957492 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:02.957496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:02.957558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:02.957696 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:02.959049 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:02.959793 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:02.960999 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:02.961413 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:02.961545 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:02.965869 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:02.965884 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:02.965900 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:02.965942 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:02.969075 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:02.969901 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:02.969909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:02.969912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:02.970071 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:02.970623 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:02.970629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:02.970632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:02.971015 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:02.971350 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:02.971633 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 20:46:02.971875 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":281,"threads_started":1}
W20260926 20:46:02.973035 7524 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48398
W20260926 20:46:02.973747 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7394: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:46:02.973897 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 7394
I20260926 20:46:03.000340 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7533
Sep 26 20:46:03 dist-test-slave-z7j8 krb5kdc[7138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:46:03.016216 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:03.001126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.5.78.3:36821)
0926 20:46:03.001645 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:03.001649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:03.001664 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:46:03.001714 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:03.001718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:03.001778 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:03.001896 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:03.003210 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:03.003953 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:03.005227 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:03.005665 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:03.005807 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:03.010383 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:03.010399 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:03.010464 (+ 65us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:03.010519 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:03.013420 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:03.014241 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:03.014249 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:03.014252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:03.014498 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:03.014920 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:03.014924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:03.014926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:03.015292 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:03.015526 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:03.015771 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 20:46:03.015950 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":270,"threads_started":1}
W20260926 20:46:03.016909 7663 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40112
W20260926 20:46:03.017584 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7533: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:46:03.017802 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 7533
I20260926 20:46:03.040881 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7151
I20260926 20:46:03.042128 7223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:03.160701 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 7151
2026-09-26T20:46:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8440 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:03 dist-test-slave-z7j8 krb5kdc[7849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:03 dist-test-slave-z7j8 krb5kdc[7849](info): set up 2 sockets
Sep 26 20:46:03 dist-test-slave-z7j8 krb5kdc[7849](info): commencing operation
krb5kdc: starting...
W20260926 20:46:04.250639 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:04 dist-test-slave-z7j8 krb5kdc[7849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:04.457412 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38963
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:38963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:04.788475 7862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:04.788848 7862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:04.789002 7862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:04.796324 7862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:04.796481 7862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:04.796588 7862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:04.796687 7862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:04.796772 7862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:04.806773 7862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:38963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:04.808122 7862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:04.809461 7862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:04.821904 7867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:04.821985 7868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:04.821929 7870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:04.822593 7862 server_base.cc:1034] running on GCE node
I20260926 20:46:04.823447 7862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:04.824997 7862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:04.826263 7862 hybrid_clock.cc:648] HybridClock initialized: now 1790455564826207 us; error 51 us; skew 500 ppm
I20260926 20:46:04.826689 7862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:04 dist-test-slave-z7j8 krb5kdc[7849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455564, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:04.829890 7862 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:04.831487 7862 webserver.cc:460] Webserver started at http://127.5.78.62:41065/ using document root <none> and password file <none>
I20260926 20:46:04.832211 7862 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:04.832361 7862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:04.832703 7862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:04.835937 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "0f3a0a93dccb4c328366f3a63c50a61c"
format_stamp: "Formatted at 2026-09-26 20:46:04 on dist-test-slave-z7j8"
server_key: "44399ec2a224a5013456fbbf38a92573"
server_key_iv: "447e242b21cd1a96688b82fd30bd6cac"
server_key_version: "encryptionkey@0"
I20260926 20:46:04.836781 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "0f3a0a93dccb4c328366f3a63c50a61c"
format_stamp: "Formatted at 2026-09-26 20:46:04 on dist-test-slave-z7j8"
server_key: "44399ec2a224a5013456fbbf38a92573"
server_key_iv: "447e242b21cd1a96688b82fd30bd6cac"
server_key_version: "encryptionkey@0"
I20260926 20:46:04.842286 7862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:46:04.845980 7877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:04.847666 7862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:04.847935 7862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "0f3a0a93dccb4c328366f3a63c50a61c"
format_stamp: "Formatted at 2026-09-26 20:46:04 on dist-test-slave-z7j8"
server_key: "44399ec2a224a5013456fbbf38a92573"
server_key_iv: "447e242b21cd1a96688b82fd30bd6cac"
server_key_version: "encryptionkey@0"
I20260926 20:46:04.848182 7862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:04.862864 7862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:04.867871 7862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:04.868218 7862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:04.889427 7862 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:38963
I20260926 20:46:04.889457 7928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:38963 every 8 connection(s)
I20260926 20:46:04.891023 7862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:04.895067 7929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:04.897336 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 7862
I20260926 20:46:04.897572 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:04.898330 5432 external_mini_cluster.cc:1428] Setting key 6e13b4e8880e8f2b1e7cd19512830f59
I20260926 20:46:04.908589 7929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c: Bootstrap starting.
I20260926 20:46:04.912832 7929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c: Neither blocks nor log segments found. Creating new log.
Sep 26 20:46:04 dist-test-slave-z7j8 krb5kdc[7849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:04.914088 7929 log.cc:824] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:04.917196 7929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c: No bootstrap required, opened a new log
I20260926 20:46:04.923496 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:04.900500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.5.78.62:38963)
0926 20:46:04.901326 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:04.901341 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:04.901386 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:46:04.903162 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:04.903186 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:04.903624 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:04.903995 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:04.905858 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:04.907351 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:04.909171 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:04.909891 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:04.910038 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:04.915126 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:04.915136 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:04.915153 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:04.915214 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:04.918669 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:04.919493 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:04.919502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:04.919509 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:04.919716 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:04.920191 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:04.920202 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:04.920213 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:04.920886 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:04.921220 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:04.921760 (+ 540us) server_negotiation.cc:299] Negotiation successful
0926 20:46:04.922270 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":371,"threads_started":1}
I20260926 20:46:04.925405 7929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38963 } }
I20260926 20:46:04.925815 7929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:04.925940 7929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3a0a93dccb4c328366f3a63c50a61c, State: Initialized, Role: FOLLOWER
I20260926 20:46:04.926610 7929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38963 } }
I20260926 20:46:04.926911 7929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:04.927040 7929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:04.927173 7929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:04.929368 7929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38963 } }
I20260926 20:46:04.929908 7929 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3a0a93dccb4c328366f3a63c50a61c; no voters:
I20260926 20:46:04.930631 7929 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:04.930733 7934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:04.931782 7934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [term 1 LEADER]: Becoming Leader. State: Replica: 0f3a0a93dccb4c328366f3a63c50a61c, State: Running, Role: LEADER
I20260926 20:46:04.932363 7934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38963 } }
I20260926 20:46:04.933079 7929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:04.935762 7935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38963 } } }
I20260926 20:46:04.935721 7936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3a0a93dccb4c328366f3a63c50a61c. Latest consensus state: current_term: 1 leader_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3a0a93dccb4c328366f3a63c50a61c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38963 } } }
I20260926 20:46:04.936264 7936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:04.936538 7935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:04.943051 7942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:04.948875 7942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:04.958989 7942 catalog_manager.cc:1347] Generated new cluster ID: 15a9c18c5fe44a92be0d73fe45b21245
I20260926 20:46:04.959144 7942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:04.985977 7942 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:04.987968 7942 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:04.999492 7942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3a0a93dccb4c328366f3a63c50a61c: Generated new TSK 0
I20260926 20:46:05.000633 7942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:46:05.058346 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38963
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:05.398567 7955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:05.398874 7955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:05.398972 7955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:05.406354 7955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:05.406579 7955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:05.406773 7955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:05.417341 7955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:05.418849 7955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:05.420248 7955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:05.434729 7960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:05.434805 7963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:05.435328 7961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:05.435855 7955 server_base.cc:1034] running on GCE node
I20260926 20:46:05.436599 7955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:05.437659 7955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:05.438875 7955 hybrid_clock.cc:648] HybridClock initialized: now 1790455565438826 us; error 43 us; skew 500 ppm
I20260926 20:46:05.439195 7955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:05 dist-test-slave-z7j8 krb5kdc[7849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455565, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:05.442595 7955 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:46:05.444296 7955 webserver.cc:460] Webserver started at http://127.5.78.1:40965/ using document root <none> and password file <none>
I20260926 20:46:05.445014 7955 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:05.445119 7955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:05.445389 7955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:05.448439 7955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "065d7ab0a85f4c6fbd7ec2d8becb0c57"
format_stamp: "Formatted at 2026-09-26 20:46:05 on dist-test-slave-z7j8"
server_key: "85d65e6861ed46ed90a79dabadcd837e"
server_key_iv: "03d22a374c4d41aa3e9d84327074b671"
server_key_version: "encryptionkey@0"
I20260926 20:46:05.449270 7955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "065d7ab0a85f4c6fbd7ec2d8becb0c57"
format_stamp: "Formatted at 2026-09-26 20:46:05 on dist-test-slave-z7j8"
server_key: "85d65e6861ed46ed90a79dabadcd837e"
server_key_iv: "03d22a374c4d41aa3e9d84327074b671"
server_key_version: "encryptionkey@0"
I20260926 20:46:05.454816 7955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:46:05.458351 7970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:05.459903 7955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:05.460045 7955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "065d7ab0a85f4c6fbd7ec2d8becb0c57"
format_stamp: "Formatted at 2026-09-26 20:46:05 on dist-test-slave-z7j8"
server_key: "85d65e6861ed46ed90a79dabadcd837e"
server_key_iv: "03d22a374c4d41aa3e9d84327074b671"
server_key_version: "encryptionkey@0"
I20260926 20:46:05.460224 7955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:05.482726 7955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:05.487188 7955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:05.487442 7955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:05.488392 7955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:05.489822 7955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:05.489900 7955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:05.489985 7955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:05.490028 7955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:05.523269 7955 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:46505
I20260926 20:46:05.523469 8082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:46505 every 8 connection(s)
I20260926 20:46:05.524595 7955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:05.530875 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 7955
I20260926 20:46:05.531152 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:05.531757 5432 external_mini_cluster.cc:1428] Setting key affc74424bc76cc7ba8db78187e7a954
Sep 26 20:46:05 dist-test-slave-z7j8 krb5kdc[7849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455565, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:05.545966 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:05.527367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:43487 (local address 127.5.78.62:38963)
0926 20:46:05.527834 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:05.527838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:05.528653 (+ 815us) server_negotiation.cc:407] Connection header received
0926 20:46:05.530500 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:05.530505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:05.530603 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:05.530728 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:05.532476 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:05.533353 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:05.535186 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:05.535950 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:05.536124 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:05.539954 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:05.539959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:05.539962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:05.539993 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:05.542747 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:05.543610 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:05.543616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:05.543619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:05.543748 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:05.544184 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:05.544191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:05.544195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:05.544472 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:05.544635 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:05.545362 (+ 727us) server_negotiation.cc:299] Negotiation successful
0926 20:46:05.545596 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":254,"threads_started":1}
I20260926 20:46:05.547003 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:05.527757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38963 (local address 127.5.78.1:43487)
0926 20:46:05.528455 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:05.528499 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:05.530188 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:05.530973 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:05.530982 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:05.531625 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:05.532275 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:05.532301 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:05.533509 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:05.533514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:05.535036 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:05.535050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:05.536219 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:05.536225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:05.536441 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:05.537290 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:05.537315 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:05.539711 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:05.542893 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:05.542903 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:05.542923 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:05.543484 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:05.543858 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:05.543863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:05.543867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:05.544068 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:05.544602 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:05.544610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:05.544989 (+ 379us) client_negotiation.cc:769] Sending connection context
0926 20:46:05.545268 (+ 279us) client_negotiation.cc:240] Negotiation successful
0926 20:46:05.545823 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":273,"threads_started":1}
I20260926 20:46:05.548520 8083 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38963
I20260926 20:46:05.548877 8083 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:46:05.549772 8083 heartbeater.cc:511] Master 127.5.78.62:38963 requested a full tablet report, sending...
I20260926 20:46:05.551985 7894 ts_manager.cc:194] Registered new tserver with Master: 065d7ab0a85f4c6fbd7ec2d8becb0c57 (127.5.78.1:46505)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:46:05.553546 7894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:43487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:46:05.578722 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38963
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:05.912463 8089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:05.912770 8089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:05.912930 8089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:05.920380 8089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:05.920507 8089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:05.920693 8089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:05.930825 8089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:05.932175 8089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:05.933504 8089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:05.947324 8094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:05.947428 8095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:05.947342 8097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:05.948432 8089 server_base.cc:1034] running on GCE node
I20260926 20:46:05.949090 8089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:05.950184 8089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:05.951409 8089 hybrid_clock.cc:648] HybridClock initialized: now 1790455565951330 us; error 74 us; skew 500 ppm
I20260926 20:46:05.951776 8089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:05 dist-test-slave-z7j8 krb5kdc[7849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455565, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:05.954985 8089 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:46:05.956364 8089 webserver.cc:460] Webserver started at http://127.5.78.2:35221/ using document root <none> and password file <none>
I20260926 20:46:05.957018 8089 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:05.957129 8089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:05.957370 8089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:05.960307 8089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "65972cd0b53a42b6abb7c29557a28a18"
format_stamp: "Formatted at 2026-09-26 20:46:05 on dist-test-slave-z7j8"
server_key: "a4814c533db7cd020b97177bb86dcfef"
server_key_iv: "92bcef6fecc744aed4e7d8317f3843ab"
server_key_version: "encryptionkey@0"
I20260926 20:46:05.961062 8089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "65972cd0b53a42b6abb7c29557a28a18"
format_stamp: "Formatted at 2026-09-26 20:46:05 on dist-test-slave-z7j8"
server_key: "a4814c533db7cd020b97177bb86dcfef"
server_key_iv: "92bcef6fecc744aed4e7d8317f3843ab"
server_key_version: "encryptionkey@0"
I20260926 20:46:05.966534 8089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:46:05.970253 8104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:05.971774 8089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:05.971954 8089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "65972cd0b53a42b6abb7c29557a28a18"
format_stamp: "Formatted at 2026-09-26 20:46:05 on dist-test-slave-z7j8"
server_key: "a4814c533db7cd020b97177bb86dcfef"
server_key_iv: "92bcef6fecc744aed4e7d8317f3843ab"
server_key_version: "encryptionkey@0"
I20260926 20:46:05.972195 8089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:05.995690 8089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:06.000520 8089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:06.000850 8089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:06.001825 8089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:06.003213 8089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:06.003283 8089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:06.003446 8089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:06.003499 8089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:06.038000 8089 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:46623
I20260926 20:46:06.038056 8216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:46623 every 8 connection(s)
I20260926 20:46:06.039417 8089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:06.048326 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8089
I20260926 20:46:06.048534 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:06.049113 5432 external_mini_cluster.cc:1428] Setting key 8eab6679179de72821bd3d519247e5c5
Sep 26 20:46:06 dist-test-slave-z7j8 krb5kdc[7849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455565, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:06.062247 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:06.042157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:47455 (local address 127.5.78.62:38963)
0926 20:46:06.042369 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:06.042376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:06.043815 (+ 1439us) server_negotiation.cc:407] Connection header received
0926 20:46:06.045675 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:06.045680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:06.045749 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:06.045843 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:06.047637 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.048576 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.050762 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.051396 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.051534 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:06.055547 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:06.055553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:06.055556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:06.055591 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:06.058333 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:06.059578 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:06.059582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:06.059585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:06.059719 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:06.060277 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:06.060281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:06.060282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:06.060536 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:06.060677 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:06.061742 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0926 20:46:06.061932 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:46:06.064095 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:06.042736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38963 (local address 127.5.78.2:47455)
0926 20:46:06.043642 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:06.043684 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:06.045381 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:06.046125 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:06.046133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:06.046812 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:06.047432 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:06.047452 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.048767 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.048773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:06.050611 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:06.050624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.051625 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.051633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:06.051902 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:06.052658 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:06.052681 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:06.055268 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:06.058577 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:06.058592 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:06.058617 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:06.059410 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:06.059842 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:06.059850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:06.059858 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:06.060152 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:06.060688 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:06.060700 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:06.061258 (+ 558us) client_negotiation.cc:769] Sending connection context
0926 20:46:06.061626 (+ 368us) client_negotiation.cc:240] Negotiation successful
0926 20:46:06.062323 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":326,"threads_started":1}
I20260926 20:46:06.065891 8217 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38963
I20260926 20:46:06.066293 8217 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:06.067368 8217 heartbeater.cc:511] Master 127.5.78.62:38963 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:46:06.069206 7894 ts_manager.cc:194] Registered new tserver with Master: 65972cd0b53a42b6abb7c29557a28a18 (127.5.78.2:46623)
I20260926 20:46:06.070055 7894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:47455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:46:06.094534 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38963
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:06.426723 8222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:06.427038 8222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:06.427135 8222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:06.434852 8222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:06.434975 8222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:06.435091 8222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:06.445390 8222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:06.446798 8222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:06.448113 8222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:06.462128 8227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:06.462095 8230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:06.462095 8228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:06.463061 8222 server_base.cc:1034] running on GCE node
I20260926 20:46:06.463706 8222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:06.464812 8222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:06.466105 8222 hybrid_clock.cc:648] HybridClock initialized: now 1790455566466014 us; error 80 us; skew 500 ppm
I20260926 20:46:06.466530 8222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:06 dist-test-slave-z7j8 krb5kdc[7849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:06.469853 8222 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:46:06.471381 8222 webserver.cc:460] Webserver started at http://127.5.78.3:44485/ using document root <none> and password file <none>
I20260926 20:46:06.472225 8222 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:06.472378 8222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:06.472708 8222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:06.475948 8222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "46f43dc8cc104f5fb4c55d6697273b11"
format_stamp: "Formatted at 2026-09-26 20:46:06 on dist-test-slave-z7j8"
server_key: "ff1766af529b7bf2b89ac303d7813078"
server_key_iv: "28af96c4d5285b3b06280483216f5224"
server_key_version: "encryptionkey@0"
I20260926 20:46:06.476748 8222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "46f43dc8cc104f5fb4c55d6697273b11"
format_stamp: "Formatted at 2026-09-26 20:46:06 on dist-test-slave-z7j8"
server_key: "ff1766af529b7bf2b89ac303d7813078"
server_key_iv: "28af96c4d5285b3b06280483216f5224"
server_key_version: "encryptionkey@0"
I20260926 20:46:06.482378 8222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:46:06.486099 8237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:06.487740 8222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:06.487962 8222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "46f43dc8cc104f5fb4c55d6697273b11"
format_stamp: "Formatted at 2026-09-26 20:46:06 on dist-test-slave-z7j8"
server_key: "ff1766af529b7bf2b89ac303d7813078"
server_key_iv: "28af96c4d5285b3b06280483216f5224"
server_key_version: "encryptionkey@0"
I20260926 20:46:06.488197 8222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:06.504541 8222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:06.509443 8222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:06.509822 8222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:06.511310 8222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:06.512648 8222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:06.512777 8222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:06.512892 8222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:06.512943 8222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:06.548834 8222 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:44951
I20260926 20:46:06.548892 8349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:44951 every 8 connection(s)
I20260926 20:46:06.550164 8222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:06.551280 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8222
I20260926 20:46:06.551481 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:06.551909 5432 external_mini_cluster.cc:1428] Setting key d53d4c8578b151d892b0e929fdab1a52
I20260926 20:46:06.556517 8083 heartbeater.cc:503] Master 127.5.78.62:38963 was elected leader, sending a full tablet report...
Sep 26 20:46:06 dist-test-slave-z7j8 krb5kdc[7849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:06.571759 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:06.553235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:37009 (local address 127.5.78.62:38963)
0926 20:46:06.553444 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:06.553451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:06.554855 (+ 1404us) server_negotiation.cc:407] Connection header received
0926 20:46:06.556818 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:06.556823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:06.556911 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:06.557035 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:06.558704 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.559454 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.561089 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.561484 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.561623 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:06.565469 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:06.565474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:06.565478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:06.565511 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:06.568427 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:06.569227 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:06.569237 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:06.569239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:06.569367 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:06.569835 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:06.569847 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:06.569850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:06.570126 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:06.570285 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:06.571163 (+ 878us) server_negotiation.cc:299] Negotiation successful
0926 20:46:06.571398 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:46:06.572772 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:06.553773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38963 (local address 127.5.78.3:37009)
0926 20:46:06.554677 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:06.554735 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:06.556483 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:06.557304 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:06.557314 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:06.557841 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:06.558527 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:06.558547 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.559630 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.559636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:06.560927 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:06.560938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:06.561724 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:06.561731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:06.561980 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:06.562807 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:06.562831 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:06.565228 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:06.568594 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:06.568610 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:06.568627 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:06.569088 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:06.569483 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:06.569487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:06.569491 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:06.569710 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:06.570264 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:06.570273 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:06.570741 (+ 468us) client_negotiation.cc:769] Sending connection context
0926 20:46:06.571036 (+ 295us) client_negotiation.cc:240] Negotiation successful
0926 20:46:06.571640 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":356,"threads_started":1}
I20260926 20:46:06.574471 8350 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38963
I20260926 20:46:06.574815 8350 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:06.575656 8350 heartbeater.cc:511] Master 127.5.78.62:38963 requested a full tablet report, sending...
I20260926 20:46:06.577461 7894 ts_manager.cc:194] Registered new tserver with Master: 46f43dc8cc104f5fb4c55d6697273b11 (127.5.78.3:44951)
I20260926 20:46:06.578341 7894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:37009
I20260926 20:46:06.580693 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 20:46:06.612460 8085 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:06.609837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.5.78.62:38963)
0926 20:46:06.610025 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:06.610030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:06.610606 (+ 576us) server_negotiation.cc:407] Connection header received
0926 20:46:06.610768 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:06.610772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:06.610837 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:06.610937 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:06.612214 (+ 1277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36020: BlockingRecv error: recv got EOF from 127.0.0.1:36020 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:46:06.616034 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7955
W20260926 20:46:06.619068 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7955: Not authorized: Client connection negotiation failed: client connection to 127.5.78.1:46505: server requires authentication, but client does not have Kerberos credentials available
I20260926 20:46:06.619257 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 7955
I20260926 20:46:06.634879 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8089
W20260926 20:46:06.637781 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8089: Not authorized: Client connection negotiation failed: client connection to 127.5.78.2:46623: server requires authentication, but client does not have Kerberos credentials available
I20260926 20:46:06.637985 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8089
I20260926 20:46:06.653537 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8222
W20260926 20:46:06.657099 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8222: Not authorized: Client connection negotiation failed: client connection to 127.5.78.3:44951: server requires authentication, but client does not have Kerberos credentials available
I20260926 20:46:06.657275 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8222
I20260926 20:46:06.672776 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 7862
I20260926 20:46:06.674063 7924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:06.775200 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 7862
2026-09-26T20:46:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3610 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:06 dist-test-slave-z7j8 krb5kdc[8367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:06 dist-test-slave-z7j8 krb5kdc[8367](info): set up 2 sockets
Sep 26 20:46:06 dist-test-slave-z7j8 krb5kdc[8367](info): commencing operation
krb5kdc: starting...
W20260926 20:46:07.874890 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:08 dist-test-slave-z7j8 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:08.072005 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:45921
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:08.398090 8380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:08.398452 8380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:08.398587 8380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:08.406241 8380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:08.406384 8380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:08.406463 8380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:08.406589 8380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:08.406661 8380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:08.416813 8380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:08.418237 8380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:08.419651 8380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:08.431460 8385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:08.431420 8388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:08.431499 8386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:08.432742 8380 server_base.cc:1034] running on GCE node
I20260926 20:46:08.433641 8380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:08.435308 8380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:08.436573 8380 hybrid_clock.cc:648] HybridClock initialized: now 1790455568436535 us; error 63 us; skew 500 ppm
I20260926 20:46:08.436976 8380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:08 dist-test-slave-z7j8 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455568, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:08.443931 8380 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:08.445689 8380 webserver.cc:460] Webserver started at http://127.5.78.62:44707/ using document root <none> and password file <none>
I20260926 20:46:08.446478 8380 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:08.446631 8380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:08.446961 8380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:08.449976 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "9037196e1e0c456999c1590bdb5ff5a7"
format_stamp: "Formatted at 2026-09-26 20:46:08 on dist-test-slave-z7j8"
server_key: "77ae4079626f766f4c93b96c818368bc"
server_key_iv: "bb162375e2d25e9fe072e951af58c226"
server_key_version: "encryptionkey@0"
I20260926 20:46:08.450878 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "9037196e1e0c456999c1590bdb5ff5a7"
format_stamp: "Formatted at 2026-09-26 20:46:08 on dist-test-slave-z7j8"
server_key: "77ae4079626f766f4c93b96c818368bc"
server_key_iv: "bb162375e2d25e9fe072e951af58c226"
server_key_version: "encryptionkey@0"
I20260926 20:46:08.456115 8380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:46:08.459896 8395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:08.461453 8380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:08.461671 8380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "9037196e1e0c456999c1590bdb5ff5a7"
format_stamp: "Formatted at 2026-09-26 20:46:08 on dist-test-slave-z7j8"
server_key: "77ae4079626f766f4c93b96c818368bc"
server_key_iv: "bb162375e2d25e9fe072e951af58c226"
server_key_version: "encryptionkey@0"
I20260926 20:46:08.461910 8380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:08.497663 8380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:08.502465 8380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:08.502803 8380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:08.522398 8380 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:45921
I20260926 20:46:08.522475 8446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:45921 every 8 connection(s)
I20260926 20:46:08.523878 8380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:08.527949 8447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:08.532294 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8380
I20260926 20:46:08.532526 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:08.533403 5432 external_mini_cluster.cc:1428] Setting key 5d846a5348455c4566b99346aba94296
I20260926 20:46:08.541589 8447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7: Bootstrap starting.
I20260926 20:46:08.545019 8447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:08.546064 8447 log.cc:824] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:46:08 dist-test-slave-z7j8 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:08.548967 8447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7: No bootstrap required, opened a new log
I20260926 20:46:08.556640 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:08.535590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.5.78.62:45921)
0926 20:46:08.536391 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:08.536405 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:08.536447 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:46:08.538057 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:08.538081 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:08.538673 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:08.539140 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:08.540600 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:08.541964 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:08.543295 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:08.543974 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:08.544162 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:08.548700 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:08.548709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:08.548725 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:08.548770 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:08.552138 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:08.552952 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:08.552960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:08.552967 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:08.553151 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:08.553579 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:08.553583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:08.553585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:08.554183 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:08.554595 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:08.555066 (+ 471us) server_negotiation.cc:299] Negotiation successful
0926 20:46:08.555537 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":356,"threads_started":1}
I20260926 20:46:08.557384 8447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9037196e1e0c456999c1590bdb5ff5a7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45921 } }
I20260926 20:46:08.557868 8447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:08.557987 8447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9037196e1e0c456999c1590bdb5ff5a7, State: Initialized, Role: FOLLOWER
I20260926 20:46:08.558723 8447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9037196e1e0c456999c1590bdb5ff5a7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45921 } }
I20260926 20:46:08.559018 8447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:08.559157 8447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:08.559338 8447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:08.561255 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:46:08.562469 8447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9037196e1e0c456999c1590bdb5ff5a7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45921 } }
I20260926 20:46:08.563091 8447 leader_election.cc:302] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9037196e1e0c456999c1590bdb5ff5a7; no voters:
I20260926 20:46:08.563748 8447 leader_election.cc:288] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:08.563870 8452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:08.566704 8452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [term 1 LEADER]: Becoming Leader. State: Replica: 9037196e1e0c456999c1590bdb5ff5a7, State: Running, Role: LEADER
I20260926 20:46:08.567340 8447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:08.567332 8452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9037196e1e0c456999c1590bdb5ff5a7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45921 } }
I20260926 20:46:08.570312 8454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9037196e1e0c456999c1590bdb5ff5a7. Latest consensus state: current_term: 1 leader_uuid: "9037196e1e0c456999c1590bdb5ff5a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9037196e1e0c456999c1590bdb5ff5a7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45921 } } }
I20260926 20:46:08.570784 8454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:08.570153 8453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9037196e1e0c456999c1590bdb5ff5a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9037196e1e0c456999c1590bdb5ff5a7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45921 } } }
I20260926 20:46:08.571323 8453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:08.579104 8459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:08.586794 8459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:08.597798 8459 catalog_manager.cc:1347] Generated new cluster ID: 2d2c43d0dd284f3ea8a2216f2b51e718
I20260926 20:46:08.598001 8459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:08.615129 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:08.605546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.5.78.62:45921)
0926 20:46:08.605738 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:08.605745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:08.606211 (+ 466us) server_negotiation.cc:407] Connection header received
0926 20:46:08.606432 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:08.606439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:08.606519 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:08.606628 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:08.607912 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:08.609035 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:08.611043 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:08.611744 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:08.611889 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:08.613131 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:08.613138 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:08.613145 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:08.614049 (+ 904us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:08.614140 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:08.614172 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:08.614288 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:08.614473 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 20:46:08.614577 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 20:46:08.626621 8459 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:08.627959 8459 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:08.642427 8459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9037196e1e0c456999c1590bdb5ff5a7: Generated new TSK 0
I20260926 20:46:08.643360 8459 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:46:08.658654 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8380
I20260926 20:46:08.660364 8442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:08.753168 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8380
2026-09-26T20:46:08Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1972 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:08 dist-test-slave-z7j8 krb5kdc[8481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:08 dist-test-slave-z7j8 krb5kdc[8481](info): set up 2 sockets
Sep 26 20:46:08 dist-test-slave-z7j8 krb5kdc[8481](info): commencing operation
krb5kdc: starting...
W20260926 20:46:09.835114 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:09 dist-test-slave-z7j8 krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:10.038033 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46217
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:46217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:10.356809 8494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:10.357137 8494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:10.357232 8494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:10.364617 8494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:10.364746 8494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:10.364822 8494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:10.364924 8494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:10.364988 8494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:10.375003 8494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:46217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:10.376459 8494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:10.377856 8494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:10.390275 8500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:10.390215 8502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:10.390280 8499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:10.391189 8494 server_base.cc:1034] running on GCE node
I20260926 20:46:10.392072 8494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:10.393653 8494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:10.394922 8494 hybrid_clock.cc:648] HybridClock initialized: now 1790455570394827 us; error 81 us; skew 500 ppm
I20260926 20:46:10.395354 8494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:10 dist-test-slave-z7j8 krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:10.398741 8494 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:10.400238 8494 webserver.cc:460] Webserver started at http://127.5.78.62:42197/ using document root <none> and password file <none>
I20260926 20:46:10.400892 8494 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:10.400988 8494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:10.401253 8494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:10.404225 8494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "2dc0fceda4e74ff389cdb96fc2acc92b"
format_stamp: "Formatted at 2026-09-26 20:46:10 on dist-test-slave-z7j8"
server_key: "f7751f73469ecb10342a92fcb483fa5a"
server_key_iv: "26647e01512134da4e4f105dcc8c1be2"
server_key_version: "encryptionkey@0"
I20260926 20:46:10.404958 8494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "2dc0fceda4e74ff389cdb96fc2acc92b"
format_stamp: "Formatted at 2026-09-26 20:46:10 on dist-test-slave-z7j8"
server_key: "f7751f73469ecb10342a92fcb483fa5a"
server_key_iv: "26647e01512134da4e4f105dcc8c1be2"
server_key_version: "encryptionkey@0"
I20260926 20:46:10.410490 8494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:46:10.414160 8509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:10.415802 8494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 20:46:10.416018 8494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "2dc0fceda4e74ff389cdb96fc2acc92b"
format_stamp: "Formatted at 2026-09-26 20:46:10 on dist-test-slave-z7j8"
server_key: "f7751f73469ecb10342a92fcb483fa5a"
server_key_iv: "26647e01512134da4e4f105dcc8c1be2"
server_key_version: "encryptionkey@0"
I20260926 20:46:10.416200 8494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:10.447254 8494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:10.452397 8494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:10.452760 8494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:10.472127 8494 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:46217
I20260926 20:46:10.472172 8560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:46217 every 8 connection(s)
I20260926 20:46:10.473613 8494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:10.477311 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8494
I20260926 20:46:10.477535 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:10.477583 8561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:10.478214 5432 external_mini_cluster.cc:1428] Setting key dd5f35596cb4e13a1e00b8d69ea9d070
I20260926 20:46:10.492383 8561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b: Bootstrap starting.
Sep 26 20:46:10 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:10.495810 8561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:10.497263 8561 log.cc:824] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:10.501032 8561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b: No bootstrap required, opened a new log
I20260926 20:46:10.504602 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:10.480442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.5.78.62:46217)
0926 20:46:10.481303 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:10.481320 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:10.481374 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:46:10.484401 (+ 3027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:10.484436 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:10.485062 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:10.485600 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:10.487026 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:10.489032 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:10.490359 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:10.491466 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:10.491664 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:10.495820 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:10.495834 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:10.495858 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:10.495913 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:10.499829 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:10.500600 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:10.500608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:10.500617 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:10.500818 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:10.501195 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:10.501200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:10.501203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:10.501946 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:10.502259 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:10.502660 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 20:46:10.503127 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":351,"threads_started":1}
I20260926 20:46:10.510571 8561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46217 } }
I20260926 20:46:10.511093 8561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:10.511238 8561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc0fceda4e74ff389cdb96fc2acc92b, State: Initialized, Role: FOLLOWER
I20260926 20:46:10.511901 8561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46217 } }
I20260926 20:46:10.512108 8561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:10.512199 8561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:10.512327 8561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:10.514448 8561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46217 } }
I20260926 20:46:10.514922 8561 leader_election.cc:302] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dc0fceda4e74ff389cdb96fc2acc92b; no voters:
I20260926 20:46:10.515437 8561 leader_election.cc:288] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:10.515583 8566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:10.516287 8566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [term 1 LEADER]: Becoming Leader. State: Replica: 2dc0fceda4e74ff389cdb96fc2acc92b, State: Running, Role: LEADER
I20260926 20:46:10.516887 8566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46217 } }
I20260926 20:46:10.517629 8561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:10.520040 8567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46217 } } }
I20260926 20:46:10.520438 8567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:10.520825 8568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dc0fceda4e74ff389cdb96fc2acc92b. Latest consensus state: current_term: 1 leader_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc0fceda4e74ff389cdb96fc2acc92b" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46217 } } }
I20260926 20:46:10.521294 8568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:10.523058 8575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:10.527281 8575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:10.535370 8575 catalog_manager.cc:1347] Generated new cluster ID: 9915c66fd1c7409c8ca68da1c6ede18c
I20260926 20:46:10.535483 8575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:10.550434 8575 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:10.552306 8575 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:10.570803 8575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dc0fceda4e74ff389cdb96fc2acc92b: Generated new TSK 0
I20260926 20:46:10.571985 8575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:46:10.622603 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46217
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:10.973256 8587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:10.973564 8587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:10.973656 8587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:10.981055 8587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:10.981190 8587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:10.981324 8587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:10.991458 8587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:10.992794 8587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:10.994469 8587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:11.008592 8593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:11.008616 8592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:11.009389 8595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:11.009936 8587 server_base.cc:1034] running on GCE node
I20260926 20:46:11.010725 8587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:11.011722 8587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:11.012923 8587 hybrid_clock.cc:648] HybridClock initialized: now 1790455571012871 us; error 41 us; skew 500 ppm
I20260926 20:46:11.013340 8587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:11 dist-test-slave-z7j8 krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:11.016933 8587 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:46:11.018669 8587 webserver.cc:460] Webserver started at http://127.5.78.1:46823/ using document root <none> and password file <none>
I20260926 20:46:11.019524 8587 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:11.019687 8587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:11.020085 8587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:11.023319 8587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "43b64101af314cce88426a33d53b7afd"
format_stamp: "Formatted at 2026-09-26 20:46:11 on dist-test-slave-z7j8"
server_key: "7a2e403949b02c05751a86e5f3c90f10"
server_key_iv: "6bfb798648aea5deb7ab42cdbe76bbe5"
server_key_version: "encryptionkey@0"
I20260926 20:46:11.024114 8587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "43b64101af314cce88426a33d53b7afd"
format_stamp: "Formatted at 2026-09-26 20:46:11 on dist-test-slave-z7j8"
server_key: "7a2e403949b02c05751a86e5f3c90f10"
server_key_iv: "6bfb798648aea5deb7ab42cdbe76bbe5"
server_key_version: "encryptionkey@0"
I20260926 20:46:11.029417 8587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:46:11.033072 8602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:11.034763 8587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:46:11.034958 8587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "43b64101af314cce88426a33d53b7afd"
format_stamp: "Formatted at 2026-09-26 20:46:11 on dist-test-slave-z7j8"
server_key: "7a2e403949b02c05751a86e5f3c90f10"
server_key_iv: "6bfb798648aea5deb7ab42cdbe76bbe5"
server_key_version: "encryptionkey@0"
I20260926 20:46:11.035178 8587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:11.053813 8587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:11.058291 8587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:11.058645 8587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:11.059638 8587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:11.061012 8587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:11.061125 8587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:11.061278 8587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:11.061362 8587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:11.098969 8587 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:40673
I20260926 20:46:11.099013 8714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:40673 every 8 connection(s)
I20260926 20:46:11.100311 8587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:11.108443 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8587
I20260926 20:46:11.108646 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:11.109299 5432 external_mini_cluster.cc:1428] Setting key 50046a13639a062f5f30accfd9e3253a
Sep 26 20:46:11 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:11.122725 8717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:11.102982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:34117 (local address 127.5.78.62:46217)
0926 20:46:11.103427 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:11.103432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:11.104654 (+ 1222us) server_negotiation.cc:407] Connection header received
0926 20:46:11.106647 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:11.106653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:11.106747 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:11.106872 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:11.108442 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.109271 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.111184 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.111582 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.111725 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:11.115683 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:11.115693 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:11.115700 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:11.115744 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:11.118802 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:11.119851 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:11.119856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:11.119858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:11.120034 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:11.120673 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:11.120677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:11.120681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:11.121000 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:11.121121 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:11.122170 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0926 20:46:11.122354 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":232,"threads_started":1}
I20260926 20:46:11.124315 8718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:11.103626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46217 (local address 127.5.78.1:34117)
0926 20:46:11.104377 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:11.104418 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:11.106283 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:11.107148 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:11.107157 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:11.107684 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:11.108264 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:11.108282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.109483 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.109487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:11.111025 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:11.111037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.111832 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.111839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:11.112070 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:11.112832 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:11.112864 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:11.115417 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:11.118977 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:11.118989 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:11.119014 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:11.119708 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:11.120190 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:11.120198 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:11.120205 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:11.120468 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:11.121165 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:11.121176 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:11.121700 (+ 524us) client_negotiation.cc:769] Sending connection context
0926 20:46:11.122019 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 20:46:11.122688 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":295,"threads_started":1}
I20260926 20:46:11.126266 8715 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46217
I20260926 20:46:11.126680 8715 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:11.127476 8715 heartbeater.cc:511] Master 127.5.78.62:46217 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:46:11.129868 8526 ts_manager.cc:194] Registered new tserver with Master: 43b64101af314cce88426a33d53b7afd (127.5.78.1:40673)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:46:11.131578 8526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:34117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:46:11.155854 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46217
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:11.490610 8721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:11.490971 8721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:11.491137 8721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:11.498814 8721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:11.498984 8721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:11.499168 8721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:11.509310 8721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:11.510831 8721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:11.512202 8721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:11.526819 8726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:11.526901 8727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:11.526865 8729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:11.527953 8721 server_base.cc:1034] running on GCE node
I20260926 20:46:11.528672 8721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:11.529739 8721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:11.530987 8721 hybrid_clock.cc:648] HybridClock initialized: now 1790455571530915 us; error 74 us; skew 500 ppm
I20260926 20:46:11.531371 8721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:11 dist-test-slave-z7j8 krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:11.534890 8721 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:46:11.536588 8721 webserver.cc:460] Webserver started at http://127.5.78.2:35575/ using document root <none> and password file <none>
I20260926 20:46:11.537320 8721 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:11.537426 8721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:11.537669 8721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:11.540969 8721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "4a2b58b825074ff6aafc0aa04ed1898a"
format_stamp: "Formatted at 2026-09-26 20:46:11 on dist-test-slave-z7j8"
server_key: "5d5c4a62418837671dae74c7ab625649"
server_key_iv: "82dcd1f5dc7925b04164bbe25ea2c7b3"
server_key_version: "encryptionkey@0"
I20260926 20:46:11.541786 8721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "4a2b58b825074ff6aafc0aa04ed1898a"
format_stamp: "Formatted at 2026-09-26 20:46:11 on dist-test-slave-z7j8"
server_key: "5d5c4a62418837671dae74c7ab625649"
server_key_iv: "82dcd1f5dc7925b04164bbe25ea2c7b3"
server_key_version: "encryptionkey@0"
I20260926 20:46:11.547572 8721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:46:11.551856 8736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:11.553503 8721 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:46:11.553680 8721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "4a2b58b825074ff6aafc0aa04ed1898a"
format_stamp: "Formatted at 2026-09-26 20:46:11 on dist-test-slave-z7j8"
server_key: "5d5c4a62418837671dae74c7ab625649"
server_key_iv: "82dcd1f5dc7925b04164bbe25ea2c7b3"
server_key_version: "encryptionkey@0"
I20260926 20:46:11.553882 8721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:11.565254 8721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:11.570158 8721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:11.570523 8721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:11.571471 8721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:11.572899 8721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:11.572975 8721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:11.573061 8721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:11.573107 8721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:11.608783 8721 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:40043
I20260926 20:46:11.608834 8848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:40043 every 8 connection(s)
I20260926 20:46:11.610175 8721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:11.615733 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8721
I20260926 20:46:11.615932 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:11.616442 5432 external_mini_cluster.cc:1428] Setting key 777660486ba21d4d37845eed81487c63
Sep 26 20:46:11 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:11.635547 8717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:11.613193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:51249 (local address 127.5.78.62:46217)
0926 20:46:11.613450 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:11.613456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:11.614864 (+ 1408us) server_negotiation.cc:407] Connection header received
0926 20:46:11.616974 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:11.616980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:11.617082 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:11.617204 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:11.619642 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.620611 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.623856 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.624320 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.624450 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:11.629246 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:11.629252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:11.629256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:11.629294 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:11.632141 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:11.633043 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:11.633048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:11.633050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:11.633183 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:11.633677 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:11.633682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:11.633684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:11.633959 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:11.634124 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:11.634966 (+ 842us) server_negotiation.cc:299] Negotiation successful
0926 20:46:11.635204 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:46:11.636623 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:11.613798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46217 (local address 127.5.78.2:51249)
0926 20:46:11.614668 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:11.614718 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:11.616650 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:11.617590 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:11.617604 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:11.618388 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:11.619384 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:11.619418 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.620814 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.620823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:11.623477 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:11.623508 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:11.624626 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:11.624635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:11.624887 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:11.625733 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:11.625780 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:11.628915 (+ 3135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:11.632342 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:11.632353 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:11.632375 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:11.632881 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:11.633314 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:11.633320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:11.633324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:11.633533 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:11.634104 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:11.634112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:11.634557 (+ 445us) client_negotiation.cc:769] Sending connection context
0926 20:46:11.634850 (+ 293us) client_negotiation.cc:240] Negotiation successful
0926 20:46:11.635454 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":348,"threads_started":1}
I20260926 20:46:11.638326 8849 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:46:11.638841 8849 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:11.639611 8849 heartbeater.cc:511] Master 127.5.78.62:46217 requested a full tablet report, sending...
I20260926 20:46:11.641297 8526 ts_manager.cc:194] Registered new tserver with Master: 4a2b58b825074ff6aafc0aa04ed1898a (127.5.78.2:40043)
I20260926 20:46:11.642599 8526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:51249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:46:11.664054 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46217
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:11.996812 8854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:11.997197 8854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:11.997392 8854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:12.004777 8854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:12.004963 8854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:12.005167 8854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:12.017144 8854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:12.018827 8854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:12.020428 8854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:12.035012 8862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:12.035102 8860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:12.035110 8859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:12.036085 8854 server_base.cc:1034] running on GCE node
I20260926 20:46:12.036809 8854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:12.037940 8854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:12.039215 8854 hybrid_clock.cc:648] HybridClock initialized: now 1790455572039131 us; error 74 us; skew 500 ppm
I20260926 20:46:12.039610 8854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:12.043205 8854 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:46:12.044922 8854 webserver.cc:460] Webserver started at http://127.5.78.3:44627/ using document root <none> and password file <none>
I20260926 20:46:12.045707 8854 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:12.045866 8854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:12.046187 8854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:12.049741 8854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "7cfca018db8544f3b8ecaa08a2a93219"
format_stamp: "Formatted at 2026-09-26 20:46:12 on dist-test-slave-z7j8"
server_key: "2ef55705891681fcc09d38a9cfa4c333"
server_key_iv: "3f8beb0b8ac28c184e7f69ad75799f18"
server_key_version: "encryptionkey@0"
I20260926 20:46:12.050853 8854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "7cfca018db8544f3b8ecaa08a2a93219"
format_stamp: "Formatted at 2026-09-26 20:46:12 on dist-test-slave-z7j8"
server_key: "2ef55705891681fcc09d38a9cfa4c333"
server_key_iv: "3f8beb0b8ac28c184e7f69ad75799f18"
server_key_version: "encryptionkey@0"
I20260926 20:46:12.056744 8854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:12.060874 8869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:12.062688 8854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:12.062906 8854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "7cfca018db8544f3b8ecaa08a2a93219"
format_stamp: "Formatted at 2026-09-26 20:46:12 on dist-test-slave-z7j8"
server_key: "2ef55705891681fcc09d38a9cfa4c333"
server_key_iv: "3f8beb0b8ac28c184e7f69ad75799f18"
server_key_version: "encryptionkey@0"
I20260926 20:46:12.063129 8854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:12.079370 8854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:12.084014 8854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:12.084345 8854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:12.085373 8854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:12.086910 8854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:12.087026 8854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:12.087182 8854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:12.087255 8854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:12.121233 8981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:42205 every 8 connection(s)
I20260926 20:46:12.121264 8854 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:42205
I20260926 20:46:12.123064 8854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:12.123734 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 8854
I20260926 20:46:12.123934 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:12.124523 5432 external_mini_cluster.cc:1428] Setting key 04df7d2fa33cabd6eab71283e58ee919
I20260926 20:46:12.135308 8715 heartbeater.cc:503] Master 127.5.78.62:46217 was elected leader, sending a full tablet report...
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:12.144413 8717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:12.125849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:38537 (local address 127.5.78.62:46217)
0926 20:46:12.126043 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:12.126050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:12.127477 (+ 1427us) server_negotiation.cc:407] Connection header received
0926 20:46:12.129230 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:12.129234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:12.129297 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:12.129386 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:12.130877 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.131608 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.133202 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.133636 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.133747 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:12.137963 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:12.137969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:12.137974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:12.138020 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:12.141345 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.142199 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.142203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.142205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.142347 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.142856 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.142860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.142862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.143114 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:12.143219 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:12.143999 (+ 780us) server_negotiation.cc:299] Negotiation successful
0926 20:46:12.144164 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:46:12.145411 8984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:12.126450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46217 (local address 127.5.78.3:38537)
0926 20:46:12.127313 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:12.127369 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:12.129002 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:12.129597 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:12.129605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:12.130102 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:12.130724 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:12.130743 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.131757 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.131762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:12.133030 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:12.133043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.133834 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.133838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:12.134069 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:12.134821 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:12.134845 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:12.137738 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:12.141528 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:12.141536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:12.141551 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:12.142075 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:12.142518 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:12.142521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:12.142524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:12.142747 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:12.143254 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:12.143260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:12.143617 (+ 357us) client_negotiation.cc:769] Sending connection context
0926 20:46:12.143888 (+ 271us) client_negotiation.cc:240] Negotiation successful
0926 20:46:12.144310 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":330,"threads_started":1}
I20260926 20:46:12.146878 8982 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46217
I20260926 20:46:12.147184 8982 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:12.147986 8982 heartbeater.cc:511] Master 127.5.78.62:46217 requested a full tablet report, sending...
I20260926 20:46:12.149405 8526 ts_manager.cc:194] Registered new tserver with Master: 7cfca018db8544f3b8ecaa08a2a93219 (127.5.78.3:42205)
I20260926 20:46:12.150218 8526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:38537
I20260926 20:46:12.152840 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:12.200863 8717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:12.186907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37522 (local address 127.5.78.62:46217)
0926 20:46:12.187095 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:12.187100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:12.187467 (+ 367us) server_negotiation.cc:407] Connection header received
0926 20:46:12.187596 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:12.187600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:12.187683 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:12.187873 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:12.189197 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.189911 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.191169 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.191582 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.191686 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:12.195669 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:12.195674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:12.195678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:12.195726 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:12.198443 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.199159 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.199163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.199165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.199307 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.199671 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.199675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.199677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.200011 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:12.200202 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:12.200480 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 20:46:12.200641 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 20:46:12.201474 8526 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37522
I20260926 20:46:12.203333 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8587
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:12.220280 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:12.204367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.5.78.1:40673)
0926 20:46:12.205028 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:12.205036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:12.205061 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:46:12.205115 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:12.205121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:12.205342 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:12.205471 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:12.206784 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.207863 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.209138 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.209762 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.209875 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:12.214154 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:12.214167 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:12.214182 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:12.214226 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:12.217179 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.217955 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.217963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.217966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.218098 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.218677 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.218681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.218683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.219072 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:12.219396 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:12.219599 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 20:46:12.219906 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":375,"threads_started":1}
W20260926 20:46:12.221673 8710 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39012
W20260926 20:46:12.222884 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8587: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:46:12.223106 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8587
I20260926 20:46:12.238749 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8721
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:46:12.256644 8994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:12.239874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.5.78.2:40043)
0926 20:46:12.240410 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:12.240418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:12.240444 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:46:12.240503 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:12.240509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:12.240758 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:12.240960 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:12.242269 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.243432 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.244744 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.245304 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.245463 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:12.250034 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:12.250049 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:12.250064 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:12.250108 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:12.253495 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.254291 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.254300 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.254303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.254508 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.254909 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.254923 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.254926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.255285 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:12.255624 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:12.255975 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 20:46:12.256275 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":289,"threads_started":1}
W20260926 20:46:12.258078 8844 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50948
W20260926 20:46:12.259580 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8721: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:46:12.259758 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8721
I20260926 20:46:12.275790 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8854
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455572, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:46:12.294135 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:12.276758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.5.78.3:42205)
0926 20:46:12.277430 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:12.277438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:12.277470 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:46:12.277531 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:12.277537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:12.277763 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:12.277939 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:12.279546 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.280665 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.282142 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:12.282896 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:12.283022 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:12.287899 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:12.287923 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:12.287940 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:12.287983 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:12.291186 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.292048 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.292057 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.292059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.292207 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:12.292594 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:12.292598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:12.292600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:12.292986 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:12.293218 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:12.293473 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 20:46:12.293685 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":353,"threads_started":1}
W20260926 20:46:12.295512 8977 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42980
W20260926 20:46:12.296676 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8854: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:46:12.296893 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8854
I20260926 20:46:12.312638 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 8494
I20260926 20:46:12.313995 8556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:12.417567 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 8494
2026-09-26T20:46:12Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3670 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8999](info): set up 2 sockets
Sep 26 20:46:12 dist-test-slave-z7j8 krb5kdc[8999](info): commencing operation
krb5kdc: starting...
W20260926 20:46:13.510890 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:13 dist-test-slave-z7j8 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:13.715404 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42855
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:32991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:14.052029 9012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:14.052423 9012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:14.052587 9012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:14.059880 9012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:14.060036 9012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:14.060144 9012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:14.060247 9012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:14.060331 9012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:14.070097 9012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:32991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:14.071470 9012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:14.072808 9012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:14.085192 9017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:14.085279 9018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:14.085301 9020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:14.086227 9012 server_base.cc:1034] running on GCE node
I20260926 20:46:14.087167 9012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:14.088737 9012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:14.089953 9012 hybrid_clock.cc:648] HybridClock initialized: now 1790455574089896 us; error 49 us; skew 500 ppm
I20260926 20:46:14.090328 9012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:14 dist-test-slave-z7j8 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:14.111310 9012 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:14.113176 9012 webserver.cc:460] Webserver started at http://127.5.78.62:35103/ using document root <none> and password file <none>
I20260926 20:46:14.113970 9012 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:14.114137 9012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:14.114552 9012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:14.117717 9012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "194eb9d2fd874ebe9c7c5a7d455167c2"
format_stamp: "Formatted at 2026-09-26 20:46:14 on dist-test-slave-z7j8"
server_key: "57e39eaa9e1197cc3f0f6e6a0ea79fba"
server_key_iv: "306c3e45eb9dc038d8a1edd2515458b8"
server_key_version: "encryptionkey@0"
I20260926 20:46:14.118671 9012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "194eb9d2fd874ebe9c7c5a7d455167c2"
format_stamp: "Formatted at 2026-09-26 20:46:14 on dist-test-slave-z7j8"
server_key: "57e39eaa9e1197cc3f0f6e6a0ea79fba"
server_key_iv: "306c3e45eb9dc038d8a1edd2515458b8"
server_key_version: "encryptionkey@0"
I20260926 20:46:14.124480 9012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:14.128305 9027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:14.129783 9012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:14.130012 9012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "194eb9d2fd874ebe9c7c5a7d455167c2"
format_stamp: "Formatted at 2026-09-26 20:46:14 on dist-test-slave-z7j8"
server_key: "57e39eaa9e1197cc3f0f6e6a0ea79fba"
server_key_iv: "306c3e45eb9dc038d8a1edd2515458b8"
server_key_version: "encryptionkey@0"
I20260926 20:46:14.130230 9012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:14.152894 9012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:14.157764 9012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:14.158136 9012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:14.177624 9078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:42855 every 8 connection(s)
I20260926 20:46:14.177594 9012 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:42855
I20260926 20:46:14.179253 9012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:14.183127 9079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:14.185519 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9012
I20260926 20:46:14.185745 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:14.186528 5432 external_mini_cluster.cc:1428] Setting key 7dc9b480b43bbde615254440248db590
I20260926 20:46:14.195252 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2: Bootstrap starting.
I20260926 20:46:14.198552 9079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:14.199589 9079 log.cc:824] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:46:14 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:14.202955 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2: No bootstrap required, opened a new log
I20260926 20:46:14.209909 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:14.188774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37908 (local address 127.5.78.62:42855)
0926 20:46:14.189537 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:14.189550 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:14.189592 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:46:14.191371 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:14.191396 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:14.191823 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:14.192290 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:14.193804 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:14.195303 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:14.196634 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:14.197381 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:14.197601 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:14.201955 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:14.201964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:14.201980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:14.202021 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:14.205576 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:14.206527 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:14.206534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:14.206557 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:14.206719 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:14.207145 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:14.207148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:14.207150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:14.207683 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:14.207900 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:14.208372 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 20:46:14.208824 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":337,"threads_started":1}
I20260926 20:46:14.211918 9079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42855 } }
I20260926 20:46:14.212297 9079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:14.212420 9079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194eb9d2fd874ebe9c7c5a7d455167c2, State: Initialized, Role: FOLLOWER
I20260926 20:46:14.213047 9079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42855 } }
I20260926 20:46:14.213258 9079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:14.213400 9079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:14.213581 9079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:14.215814 9079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42855 } }
I20260926 20:46:14.216380 9079 leader_election.cc:302] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 194eb9d2fd874ebe9c7c5a7d455167c2; no voters:
I20260926 20:46:14.216986 9079 leader_election.cc:288] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:14.217152 9084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:14.218016 9084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [term 1 LEADER]: Becoming Leader. State: Replica: 194eb9d2fd874ebe9c7c5a7d455167c2, State: Running, Role: LEADER
I20260926 20:46:14.218614 9084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42855 } }
I20260926 20:46:14.219234 9079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:14.222254 9085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42855 } } }
I20260926 20:46:14.222679 9085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:14.224758 9086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 194eb9d2fd874ebe9c7c5a7d455167c2. Latest consensus state: current_term: 1 leader_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194eb9d2fd874ebe9c7c5a7d455167c2" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42855 } } }
I20260926 20:46:14.225165 9086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:14.228353 9093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:14.233893 9093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:14.242926 9093 catalog_manager.cc:1347] Generated new cluster ID: 966e0eff116742d28832c3ec9483fe2c
I20260926 20:46:14.243083 9093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:14.263988 9093 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:14.265676 9093 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:14.279788 9093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 194eb9d2fd874ebe9c7c5a7d455167c2: Generated new TSK 0
I20260926 20:46:14.280921 9093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:46:14.338560 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42855
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:32991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:14.679593 9105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:14.679889 9105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:14.679984 9105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:14.687685 9105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:14.687808 9105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:14.687922 9105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:14.697909 9105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:32991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:14.699301 9105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:14.700608 9105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:14.714093 9111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:14.714268 9110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:14.714093 9113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:14.714918 9105 server_base.cc:1034] running on GCE node
I20260926 20:46:14.715569 9105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:14.716619 9105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:14.717854 9105 hybrid_clock.cc:648] HybridClock initialized: now 1790455574717783 us; error 67 us; skew 500 ppm
I20260926 20:46:14.718190 9105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:14 dist-test-slave-z7j8 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:14.721558 9105 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:46:14.723302 9105 webserver.cc:460] Webserver started at http://127.5.78.1:36815/ using document root <none> and password file <none>
I20260926 20:46:14.724035 9105 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:14.724197 9105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:14.724530 9105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:14.727964 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "b75c449935ba473cb59b5866cbdd3197"
format_stamp: "Formatted at 2026-09-26 20:46:14 on dist-test-slave-z7j8"
server_key: "f0b9b09547d686fa167fc0a92a3ce0dc"
server_key_iv: "224cbeda7662d7f2456894d9291776e7"
server_key_version: "encryptionkey@0"
I20260926 20:46:14.728847 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "b75c449935ba473cb59b5866cbdd3197"
format_stamp: "Formatted at 2026-09-26 20:46:14 on dist-test-slave-z7j8"
server_key: "f0b9b09547d686fa167fc0a92a3ce0dc"
server_key_iv: "224cbeda7662d7f2456894d9291776e7"
server_key_version: "encryptionkey@0"
I20260926 20:46:14.734522 9105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 20:46:14.738389 9120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:14.740067 9105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:14.740293 9105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "b75c449935ba473cb59b5866cbdd3197"
format_stamp: "Formatted at 2026-09-26 20:46:14 on dist-test-slave-z7j8"
server_key: "f0b9b09547d686fa167fc0a92a3ce0dc"
server_key_iv: "224cbeda7662d7f2456894d9291776e7"
server_key_version: "encryptionkey@0"
I20260926 20:46:14.740541 9105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:14.762272 9105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:14.766964 9105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:14.767313 9105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:14.768407 9105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:14.769793 9105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:14.769903 9105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:14.770061 9105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:14.770133 9105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:14.806161 9105 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:44247
I20260926 20:46:14.806223 9232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:44247 every 8 connection(s)
I20260926 20:46:14.807542 9105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:14.809082 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9105
I20260926 20:46:14.809310 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:14.809903 5432 external_mini_cluster.cc:1428] Setting key da939abf6dfcacd03c55ea830016caf6
Sep 26 20:46:14 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:46:14.832546 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:14.810247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:37045 (local address 127.5.78.62:42855)
0926 20:46:14.810791 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:14.810805 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:14.812026 (+ 1221us) server_negotiation.cc:407] Connection header received
0926 20:46:14.816329 (+ 4303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:14.816337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:14.816426 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:14.816546 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:14.818632 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:14.819892 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:14.821647 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:14.822378 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:14.822555 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:14.826176 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:14.826185 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:14.826192 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:14.826239 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:14.829227 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:14.830092 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:14.830097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:14.830100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:14.830219 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:14.830783 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:14.830791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:14.830797 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:14.831076 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:14.831247 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:14.831967 (+ 720us) server_negotiation.cc:299] Negotiation successful
0926 20:46:14.832189 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":293,"threads_started":1}
I20260926 20:46:14.833645 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:14.811005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42855 (local address 127.5.78.1:37045)
0926 20:46:14.811863 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:14.811926 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:14.816036 (+ 4110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:14.816870 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:14.816884 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:14.817743 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:14.818382 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:14.818399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:14.820054 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:14.820058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:14.821491 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:14.821503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:14.822617 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:14.822622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:14.822824 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:14.823526 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:14.823553 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:14.825867 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:14.829396 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:14.829405 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:14.829421 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:14.829977 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:14.830362 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:14.830370 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:14.830374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:14.830676 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:14.831212 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:14.831229 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:14.831592 (+ 363us) client_negotiation.cc:769] Sending connection context
0926 20:46:14.831869 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 20:46:14.832439 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":321,"threads_started":1}
I20260926 20:46:14.835253 9233 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42855
I20260926 20:46:14.835569 9233 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:14.836313 9233 heartbeater.cc:511] Master 127.5.78.62:42855 requested a full tablet report, sending...
I20260926 20:46:14.838578 9044 ts_manager.cc:194] Registered new tserver with Master: b75c449935ba473cb59b5866cbdd3197 (127.5.78.1:44247)
I20260926 20:46:14.840260 9044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:37045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:46:14.853868 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42855
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:32991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:15.199076 9239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:15.199425 9239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:15.199591 9239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:15.207075 9239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:15.207245 9239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:15.207417 9239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:15.217408 9239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:32991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:15.218874 9239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:15.220409 9239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:15.234470 9244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:15.234377 9247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:15.234445 9245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:15.235219 9239 server_base.cc:1034] running on GCE node
I20260926 20:46:15.235813 9239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:15.236768 9239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:15.237962 9239 hybrid_clock.cc:648] HybridClock initialized: now 1790455575237921 us; error 34 us; skew 500 ppm
I20260926 20:46:15.238296 9239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:15.241458 9239 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:46:15.243095 9239 webserver.cc:460] Webserver started at http://127.5.78.2:42179/ using document root <none> and password file <none>
I20260926 20:46:15.243773 9239 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:15.243914 9239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:15.244218 9239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:15.247220 9239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "9623e3684749405c8dc595d7ac4980c8"
format_stamp: "Formatted at 2026-09-26 20:46:15 on dist-test-slave-z7j8"
server_key: "2a14ca4bef840823d296394773b572e6"
server_key_iv: "775914e2ab6c96c6954fb6d7a862d4f7"
server_key_version: "encryptionkey@0"
I20260926 20:46:15.247975 9239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "9623e3684749405c8dc595d7ac4980c8"
format_stamp: "Formatted at 2026-09-26 20:46:15 on dist-test-slave-z7j8"
server_key: "2a14ca4bef840823d296394773b572e6"
server_key_iv: "775914e2ab6c96c6954fb6d7a862d4f7"
server_key_version: "encryptionkey@0"
I20260926 20:46:15.252918 9239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:15.256237 9254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:15.257493 9239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:15.257653 9239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "9623e3684749405c8dc595d7ac4980c8"
format_stamp: "Formatted at 2026-09-26 20:46:15 on dist-test-slave-z7j8"
server_key: "2a14ca4bef840823d296394773b572e6"
server_key_iv: "775914e2ab6c96c6954fb6d7a862d4f7"
server_key_version: "encryptionkey@0"
I20260926 20:46:15.257863 9239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:15.287523 9239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:15.292038 9239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:15.292327 9239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:15.293321 9239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:15.294677 9239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:15.294749 9239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:15.294831 9239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:15.294881 9239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:15.328132 9239 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:33841
I20260926 20:46:15.328173 9366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:33841 every 8 connection(s)
I20260926 20:46:15.329413 9239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:15.334026 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9239
I20260926 20:46:15.334228 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:15.334813 5432 external_mini_cluster.cc:1428] Setting key 003ee061c5ae2209f8bc136d599f58cc
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:15.352450 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.332150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:56509 (local address 127.5.78.62:42855)
0926 20:46:15.332325 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:15.332330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:15.333547 (+ 1217us) server_negotiation.cc:407] Connection header received
0926 20:46:15.335666 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:15.335673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:15.335761 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:15.335885 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:15.337909 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.339150 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.340911 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.341683 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.341836 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.346566 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:15.346573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:15.346576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:15.346623 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:15.349250 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.350022 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.350027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.350031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.350166 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.350705 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.350709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.350712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.350970 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:15.351146 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:15.351887 (+ 741us) server_negotiation.cc:299] Negotiation successful
0926 20:46:15.352113 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:46:15.353557 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.332652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42855 (local address 127.5.78.2:56509)
0926 20:46:15.333391 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:15.333434 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:15.335329 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:15.336211 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:15.336224 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:15.337033 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:15.337702 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:15.337723 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.339347 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.339351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:15.340739 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:15.340754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.341969 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.341979 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:15.342227 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.343128 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:15.343166 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:15.346278 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:15.349420 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:15.349429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:15.349450 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:15.349892 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:15.350286 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:15.350291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:15.350294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:15.350571 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:15.351116 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:15.351125 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:15.351508 (+ 383us) client_negotiation.cc:769] Sending connection context
0926 20:46:15.351778 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 20:46:15.352355 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":279,"threads_started":1}
I20260926 20:46:15.355104 9367 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42855
I20260926 20:46:15.355477 9367 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:46:15.356374 9367 heartbeater.cc:511] Master 127.5.78.62:42855 requested a full tablet report, sending...
I20260926 20:46:15.357945 9044 ts_manager.cc:194] Registered new tserver with Master: 9623e3684749405c8dc595d7ac4980c8 (127.5.78.2:33841)
I20260926 20:46:15.358825 9044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:56509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:46:15.382774 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42855
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:32991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:15.712280 9372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:15.712576 9372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:15.712669 9372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:15.719928 9372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:15.720052 9372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:15.720292 9372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:15.730085 9372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:32991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:15.731488 9372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:15.732792 9372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:15.746357 9377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:15.746481 9378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:15.746399 9380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:15.747578 9372 server_base.cc:1034] running on GCE node
I20260926 20:46:15.748286 9372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:15.749280 9372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:15.750519 9372 hybrid_clock.cc:648] HybridClock initialized: now 1790455575750471 us; error 39 us; skew 500 ppm
I20260926 20:46:15.750859 9372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:15.753942 9372 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:46:15.755692 9372 webserver.cc:460] Webserver started at http://127.5.78.3:37273/ using document root <none> and password file <none>
I20260926 20:46:15.756412 9372 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:15.756559 9372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:15.756872 9372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:15.760010 9372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "510d3eaae25b4a73be5c6c19fdc473be"
format_stamp: "Formatted at 2026-09-26 20:46:15 on dist-test-slave-z7j8"
server_key: "e17bbe5c2aa06872fcc0029b52892008"
server_key_iv: "3aa40433e168d5a276f6dec306c17707"
server_key_version: "encryptionkey@0"
I20260926 20:46:15.760926 9372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "510d3eaae25b4a73be5c6c19fdc473be"
format_stamp: "Formatted at 2026-09-26 20:46:15 on dist-test-slave-z7j8"
server_key: "e17bbe5c2aa06872fcc0029b52892008"
server_key_iv: "3aa40433e168d5a276f6dec306c17707"
server_key_version: "encryptionkey@0"
I20260926 20:46:15.766912 9372 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 20:46:15.770892 9387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:15.772517 9372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:15.772717 9372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "510d3eaae25b4a73be5c6c19fdc473be"
format_stamp: "Formatted at 2026-09-26 20:46:15 on dist-test-slave-z7j8"
server_key: "e17bbe5c2aa06872fcc0029b52892008"
server_key_iv: "3aa40433e168d5a276f6dec306c17707"
server_key_version: "encryptionkey@0"
I20260926 20:46:15.772943 9372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:15.810544 9372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:15.815408 9372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:15.815721 9372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:15.816745 9372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:15.818109 9372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:15.818236 9372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:15.818388 9372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:15.818490 9372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:46:15.843497 9233 heartbeater.cc:503] Master 127.5.78.62:42855 was elected leader, sending a full tablet report...
I20260926 20:46:15.853462 9372 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38439
I20260926 20:46:15.853535 9499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38439 every 8 connection(s)
I20260926 20:46:15.855067 9372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:15.863142 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9372
I20260926 20:46:15.863391 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:15.863806 5432 external_mini_cluster.cc:1428] Setting key cb519476008a4258d6ea28b178a30a22
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:15.877990 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.857682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:35301 (local address 127.5.78.62:42855)
0926 20:46:15.858113 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:15.858118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:15.859367 (+ 1249us) server_negotiation.cc:407] Connection header received
0926 20:46:15.861358 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:15.861365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:15.861457 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:15.861613 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:15.863509 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.864410 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.866804 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.867320 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.867433 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.871312 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:15.871317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:15.871321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:15.871358 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:15.874128 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.875152 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.875157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.875159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.875322 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.875843 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.875847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.875849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.876133 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:15.876297 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:15.877183 (+ 886us) server_negotiation.cc:299] Negotiation successful
0926 20:46:15.877440 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":204,"threads_started":1}
I20260926 20:46:15.879253 9503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.858329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42855 (local address 127.5.78.3:35301)
0926 20:46:15.859181 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:15.859223 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:15.860992 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:15.861889 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:15.861899 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:15.862650 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:15.863249 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:15.863272 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.864802 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.864806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:15.866620 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:15.866654 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.867583 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.867589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:15.867831 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.868519 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:15.868543 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:15.870994 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:15.874346 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:15.874355 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:15.874372 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:15.874970 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:15.875457 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:15.875461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:15.875464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:15.875685 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:15.876316 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:15.876323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:15.876784 (+ 461us) client_negotiation.cc:769] Sending connection context
0926 20:46:15.877061 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 20:46:15.877641 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":336,"threads_started":1}
I20260926 20:46:15.880834 9500 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42855
I20260926 20:46:15.881170 9500 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:15.881973 9500 heartbeater.cc:511] Master 127.5.78.62:42855 requested a full tablet report, sending...
I20260926 20:46:15.883541 9044 ts_manager.cc:194] Registered new tserver with Master: 510d3eaae25b4a73be5c6c19fdc473be (127.5.78.3:38439)
I20260926 20:46:15.884368 9044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:35301
I20260926 20:46:15.892704 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:15.941569 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.926891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.5.78.1:44247)
0926 20:46:15.927489 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:15.927496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:15.927523 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:46:15.927581 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:15.927588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:15.927795 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:15.927935 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:15.929285 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.930286 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.931498 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.932063 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.932161 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.936036 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:15.936046 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:15.936060 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:15.936097 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:15.938880 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.939685 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.939694 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.939696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.939825 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.940240 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.940243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.940245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.940581 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:15.940802 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:15.941079 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 20:46:15.941279 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":282,"threads_started":1}
Sep 26 20:46:15 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:15.960456 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.946051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37920 (local address 127.5.78.62:42855)
0926 20:46:15.946289 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:15.946296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:15.946544 (+ 248us) server_negotiation.cc:407] Connection header received
0926 20:46:15.946669 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:15.946673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:15.946737 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:15.946836 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:15.948192 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.948943 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.950347 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.950851 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.950968 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.954962 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:15.954968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:15.954971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:15.955009 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:15.957662 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.958371 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.958379 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.958385 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.958570 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.958987 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.958993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.958997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.959335 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:15.959612 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:15.959901 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 20:46:15.960132 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260926 20:46:15.961052 9044 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37920
I20260926 20:46:15.962482 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9105
I20260926 20:46:15.975656 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:15.963479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.5.78.1:44247)
0926 20:46:15.963683 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:15.963689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:15.963935 (+ 246us) server_negotiation.cc:407] Connection header received
0926 20:46:15.964029 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:15.964033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:15.964126 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:15.964213 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:15.965416 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.966099 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.967386 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:15.967815 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:15.967922 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:15.970329 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:15.970335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:15.970338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:15.970376 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:15.973106 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.973781 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.973785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.973787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.973906 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:15.974280 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:15.974283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:15.974284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:15.974588 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:15.974762 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:15.975108 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 20:46:15.975285 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:46:15.976318 9228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:16.061092 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9105
I20260926 20:46:16.077078 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9239
Sep 26 20:46:16 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:46:16.094590 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:16.078398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.5.78.2:33841)
0926 20:46:16.078931 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:16.078938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:16.078963 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:46:16.079021 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:16.079027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:16.079242 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:16.079428 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:16.080607 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:16.081870 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:16.083490 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:16.084086 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:16.084282 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:16.088618 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:16.088630 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:16.088646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:16.088684 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:16.091626 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:16.092374 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:16.092382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:16.092385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:16.092512 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:16.092897 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:16.092903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:16.092906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:16.093249 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:16.093559 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:16.093866 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 20:46:16.094128 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":235,"threads_started":1}
I20260926 20:46:16.095896 9362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:16.179858 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9239
I20260926 20:46:16.199847 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9372
Sep 26 20:46:16 dist-test-slave-z7j8 krb5kdc[8999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:46:16.216838 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:16.200930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.5.78.3:38439)
0926 20:46:16.201519 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:16.201527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:16.201557 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:46:16.201619 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:16.201625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:16.201849 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:16.202017 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:16.203426 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:16.204478 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:16.205783 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:16.206370 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:16.206668 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:16.211216 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:16.211229 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:16.211246 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:16.211287 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:16.214147 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:16.214983 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:16.214991 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:16.214994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:16.215124 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:16.215495 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:16.215498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:16.215500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:16.215834 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:16.216074 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:16.216339 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 20:46:16.216560 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":318,"threads_started":1}
I20260926 20:46:16.218154 9495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:16.301391 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9372
I20260926 20:46:16.316828 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9012
I20260926 20:46:16.318076 9074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:16.419785 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9012
2026-09-26T20:46:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4001 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:16 dist-test-slave-z7j8 krb5kdc[9520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:16 dist-test-slave-z7j8 krb5kdc[9520](info): set up 2 sockets
Sep 26 20:46:16 dist-test-slave-z7j8 krb5kdc[9520](info): commencing operation
krb5kdc: starting...
W20260926 20:46:17.506910 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.004s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:17 dist-test-slave-z7j8 krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:17.719049 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:45685
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:45685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:18.043440 9533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:18.043756 9533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:18.043850 9533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:18.051299 9533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:18.051429 9533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:18.051473 9533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:18.051530 9533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:18.051579 9533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:18.061635 9533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:45685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:45685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:18.063135 9533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:18.064468 9533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:18.077132 9538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:18.077195 9541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:18.077174 9539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:18.078977 9533 server_base.cc:1034] running on GCE node
I20260926 20:46:18.079867 9533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:18.081421 9533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:18.082660 9533 hybrid_clock.cc:648] HybridClock initialized: now 1790455578082597 us; error 68 us; skew 500 ppm
I20260926 20:46:18.083072 9533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:18 dist-test-slave-z7j8 krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455578, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:18.086724 9533 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:18.087963 9533 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:18.088065 9533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:18.088356 9533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:18.091645 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "c1e228d578c64e7d8b38c4699aea8d29"
format_stamp: "Formatted at 2026-09-26 20:46:18 on dist-test-slave-z7j8"
server_key: "1ebcaf09e57b92d5253676fd81457762"
server_key_iv: "f57294038ea193516c4adca59e065784"
server_key_version: "encryptionkey@0"
I20260926 20:46:18.092494 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "c1e228d578c64e7d8b38c4699aea8d29"
format_stamp: "Formatted at 2026-09-26 20:46:18 on dist-test-slave-z7j8"
server_key: "1ebcaf09e57b92d5253676fd81457762"
server_key_iv: "f57294038ea193516c4adca59e065784"
server_key_version: "encryptionkey@0"
I20260926 20:46:18.098232 9533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:46:18.101861 9547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:18.103274 9533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:18.103469 9533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c1e228d578c64e7d8b38c4699aea8d29"
format_stamp: "Formatted at 2026-09-26 20:46:18 on dist-test-slave-z7j8"
server_key: "1ebcaf09e57b92d5253676fd81457762"
server_key_iv: "f57294038ea193516c4adca59e065784"
server_key_version: "encryptionkey@0"
I20260926 20:46:18.103703 9533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:18.123845 9533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:18.128454 9533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:18.128746 9533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:18.147356 9533 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:45685
I20260926 20:46:18.147378 9598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:45685 every 8 connection(s)
I20260926 20:46:18.148300 9533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:18.151973 9599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:18.158680 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9533
I20260926 20:46:18.158901 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:18.159533 5432 external_mini_cluster.cc:1428] Setting key 34968523cf51b8ff0f1c5cd7ab6f5d48
I20260926 20:46:18.162462 9599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29: Bootstrap starting.
I20260926 20:46:18.165796 9599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:18.167246 9599 log.cc:824] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:18.170372 9599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29: No bootstrap required, opened a new log
Sep 26 20:46:18 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:18.181037 9599 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e228d578c64e7d8b38c4699aea8d29" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45685 } }
I20260926 20:46:18.181602 9599 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:18.181708 9599 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e228d578c64e7d8b38c4699aea8d29, State: Initialized, Role: FOLLOWER
I20260926 20:46:18.182468 9599 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e228d578c64e7d8b38c4699aea8d29" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45685 } }
I20260926 20:46:18.182736 9599 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:18.182873 9599 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:18.183077 9599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:18.185529 9599 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e228d578c64e7d8b38c4699aea8d29" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45685 } }
I20260926 20:46:18.185644 9602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:18.161461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.5.78.62:45685)
0926 20:46:18.162363 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:18.162374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:18.162464 (+ 90us) server_negotiation.cc:407] Connection header received
0926 20:46:18.164238 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:18.164265 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:18.164640 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:18.164951 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:18.166857 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:18.168691 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:18.170482 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:18.171295 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:18.171434 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:18.176113 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:18.176125 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:18.176150 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:18.176214 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:18.180628 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:18.181662 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:18.181674 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:18.181683 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:18.181897 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:18.182376 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:18.182383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:18.182387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:18.183201 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:18.183551 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:18.184107 (+ 556us) server_negotiation.cc:299] Negotiation successful
0926 20:46:18.184585 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":394,"threads_started":1}
I20260926 20:46:18.186075 9599 leader_election.cc:302] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1e228d578c64e7d8b38c4699aea8d29; no voters:
I20260926 20:46:18.186795 9599 leader_election.cc:288] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:18.186898 9604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:18.187580 9604 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [term 1 LEADER]: Becoming Leader. State: Replica: c1e228d578c64e7d8b38c4699aea8d29, State: Running, Role: LEADER
I20260926 20:46:18.188362 9604 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e228d578c64e7d8b38c4699aea8d29" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45685 } }
I20260926 20:46:18.190851 9599 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:18.191354 9605 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e228d578c64e7d8b38c4699aea8d29. Latest consensus state: current_term: 1 leader_uuid: "c1e228d578c64e7d8b38c4699aea8d29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e228d578c64e7d8b38c4699aea8d29" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45685 } } }
I20260926 20:46:18.191784 9605 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:18.192440 9606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1e228d578c64e7d8b38c4699aea8d29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e228d578c64e7d8b38c4699aea8d29" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45685 } } }
I20260926 20:46:18.192905 9606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:18.195403 9609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:18.201694 9609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:18.211285 9609 catalog_manager.cc:1347] Generated new cluster ID: 86818c96916344cf82380da3c04ebf2e
I20260926 20:46:18.211431 9609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:18.240315 9609 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:18.241739 9609 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:18.252624 9609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29: Generated new TSK 0
I20260926 20:46:18.253433 9609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:46:18.302783 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45685
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:18.635290 9625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:18.635596 9625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:18.635752 9625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:18.643326 9625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:18.643447 9625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:18.643641 9625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:18.653671 9625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:18.655125 9625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:18.656551 9625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:18.670027 9630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:18.670152 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:18.670174 9631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:18.676405 9625 server_base.cc:1034] running on GCE node
I20260926 20:46:18.677163 9625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:18.678236 9625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:18.679514 9625 hybrid_clock.cc:648] HybridClock initialized: now 1790455578679444 us; error 81 us; skew 500 ppm
I20260926 20:46:18.679908 9625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:18 dist-test-slave-z7j8 krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455578, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:18.683321 9625 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:46:18.684712 9625 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:18.684862 9625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:18.685194 9625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:18.688324 9625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2"
format_stamp: "Formatted at 2026-09-26 20:46:18 on dist-test-slave-z7j8"
server_key: "a5c67e990765e042a3c059242678075a"
server_key_iv: "3e8277d756a2133d713265b5bcc46b5f"
server_key_version: "encryptionkey@0"
I20260926 20:46:18.689222 9625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2"
format_stamp: "Formatted at 2026-09-26 20:46:18 on dist-test-slave-z7j8"
server_key: "a5c67e990765e042a3c059242678075a"
server_key_iv: "3e8277d756a2133d713265b5bcc46b5f"
server_key_version: "encryptionkey@0"
I20260926 20:46:18.694876 9625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:46:18.698661 9639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:18.700263 9625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:18.700481 9625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2"
format_stamp: "Formatted at 2026-09-26 20:46:18 on dist-test-slave-z7j8"
server_key: "a5c67e990765e042a3c059242678075a"
server_key_iv: "3e8277d756a2133d713265b5bcc46b5f"
server_key_version: "encryptionkey@0"
I20260926 20:46:18.700745 9625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:18.717532 9625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:18.722236 9625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:18.722599 9625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:18.723649 9625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:18.725064 9625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:18.725184 9625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:18.725315 9625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:18.725406 9625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:18.758973 9625 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:38993
I20260926 20:46:18.759035 9751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:38993 every 8 connection(s)
I20260926 20:46:18.759829 9625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:18.762511 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9625
I20260926 20:46:18.762779 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:18.763428 5432 external_mini_cluster.cc:1428] Setting key 8fec54b32d4fca6889ea730e0c522d70
Sep 26 20:46:18 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455578, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:18.780010 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:18.762391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:37029 (local address 127.5.78.62:45685)
0926 20:46:18.762894 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:18.762900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:18.764310 (+ 1410us) server_negotiation.cc:407] Connection header received
0926 20:46:18.765798 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:18.765803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:18.765900 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:18.766015 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:18.767579 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:18.768381 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:18.770017 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:18.770479 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:18.770617 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:18.774173 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:18.774178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:18.774181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:18.774214 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:18.776806 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:18.777635 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:18.777643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:18.777651 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:18.777769 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:18.778197 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:18.778202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:18.778206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:18.778558 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:18.778726 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:18.779433 (+ 707us) server_negotiation.cc:299] Negotiation successful
0926 20:46:18.779652 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":226,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:46:18.781073 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:18.763000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:45685 (local address 127.5.78.1:37029)
0926 20:46:18.763805 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:18.763860 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:18.765522 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:18.766252 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:18.766261 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:18.766855 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:18.767414 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:18.767430 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:18.768551 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:18.768555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:18.769854 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:18.769866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:18.770722 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:18.770730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:18.770960 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:18.771683 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:18.771710 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:18.773908 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:18.776992 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:18.777001 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:18.777022 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:18.777483 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:18.777884 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:18.777889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:18.777893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:18.778084 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:18.778704 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:18.778713 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:18.779065 (+ 352us) client_negotiation.cc:769] Sending connection context
0926 20:46:18.779338 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 20:46:18.779906 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":346,"threads_started":1}
I20260926 20:46:18.782678 9752 heartbeater.cc:347] Connected to a master server at 127.5.78.62:45685
I20260926 20:46:18.783025 9752 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:46:18.783782 9752 heartbeater.cc:511] Master 127.5.78.62:45685 requested a full tablet report, sending...
I20260926 20:46:18.786157 9564 ts_manager.cc:194] Registered new tserver with Master: 1cb94252dfe34b9c99cb9ce7b240e2c2 (127.5.78.1:38993)
I20260926 20:46:18.788353 9564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:37029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:46:18.809701 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45685
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:19.145376 9758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:19.145766 9758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:19.145951 9758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:19.153661 9758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:19.153851 9758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:19.154052 9758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:19.164609 9758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:19.166026 9758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:19.167537 9758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:19.180721 9766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:19.180763 9764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:19.181030 9763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:19.181784 9758 server_base.cc:1034] running on GCE node
I20260926 20:46:19.182581 9758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:19.183568 9758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:19.184793 9758 hybrid_clock.cc:648] HybridClock initialized: now 1790455579184720 us; error 70 us; skew 500 ppm
I20260926 20:46:19.185155 9758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:19 dist-test-slave-z7j8 krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455579, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:19.193166 9758 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:46:19.194602 9758 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:19.194734 9758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:19.195079 9758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:19.198305 9758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "d7bef08c4ff046d785bb609f41ad30c5"
format_stamp: "Formatted at 2026-09-26 20:46:19 on dist-test-slave-z7j8"
server_key: "253cd752e33dee6ddc0d4e823fdc48f0"
server_key_iv: "a74bb232c08daf24e3212f9c8a0918bb"
server_key_version: "encryptionkey@0"
I20260926 20:46:19.199182 9758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "d7bef08c4ff046d785bb609f41ad30c5"
format_stamp: "Formatted at 2026-09-26 20:46:19 on dist-test-slave-z7j8"
server_key: "253cd752e33dee6ddc0d4e823fdc48f0"
server_key_iv: "a74bb232c08daf24e3212f9c8a0918bb"
server_key_version: "encryptionkey@0"
I20260926 20:46:19.204303 9758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:19.208001 9772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:19.209357 9758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:19.209523 9758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "d7bef08c4ff046d785bb609f41ad30c5"
format_stamp: "Formatted at 2026-09-26 20:46:19 on dist-test-slave-z7j8"
server_key: "253cd752e33dee6ddc0d4e823fdc48f0"
server_key_iv: "a74bb232c08daf24e3212f9c8a0918bb"
server_key_version: "encryptionkey@0"
I20260926 20:46:19.209722 9758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:19.234910 9758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:19.240280 9758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:19.240597 9758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:19.241631 9758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:19.243199 9758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:19.243331 9758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:19.243472 9758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:19.243546 9758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:19.278546 9758 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:37253
I20260926 20:46:19.278635 9884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:37253 every 8 connection(s)
I20260926 20:46:19.279438 9758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:19.289616 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9758
I20260926 20:46:19.289834 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:19.290269 5432 external_mini_cluster.cc:1428] Setting key 0f16fd78c917c447f62764a815f662da
Sep 26 20:46:19 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455579, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:19.300892 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.282094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:45099 (local address 127.5.78.62:45685)
0926 20:46:19.282565 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:19.282569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:19.283578 (+ 1009us) server_negotiation.cc:407] Connection header received
0926 20:46:19.285447 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:19.285453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:19.285543 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:19.285678 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:19.287492 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.288213 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.289939 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.290481 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.290614 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.294787 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:19.294793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:19.294796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:19.294831 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:19.297580 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:19.298448 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:19.298455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:19.298458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:19.298606 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:19.299076 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:19.299086 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:19.299088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:19.299352 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:19.299514 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:19.300251 (+ 737us) server_negotiation.cc:299] Negotiation successful
0926 20:46:19.300485 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":210,"threads_started":1}
I20260926 20:46:19.301976 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.282743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:45685 (local address 127.5.78.2:45099)
0926 20:46:19.283427 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:19.283464 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:19.285135 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:19.285940 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:19.285949 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:19.286673 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:19.287285 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.287304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.288389 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.288395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.289755 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.289777 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.290726 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.290733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.291064 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.291857 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:19.291897 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:19.294529 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:19.297757 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:19.297768 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:19.297788 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:19.298272 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:19.298728 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:19.298733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:19.298737 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:19.298947 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:19.299493 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:19.299502 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:19.299875 (+ 373us) client_negotiation.cc:769] Sending connection context
0926 20:46:19.300158 (+ 283us) client_negotiation.cc:240] Negotiation successful
0926 20:46:19.300763 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":277,"threads_started":1}
I20260926 20:46:19.303721 9885 heartbeater.cc:347] Connected to a master server at 127.5.78.62:45685
I20260926 20:46:19.304129 9885 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:19.304911 9885 heartbeater.cc:511] Master 127.5.78.62:45685 requested a full tablet report, sending...
I20260926 20:46:19.306820 9564 ts_manager.cc:194] Registered new tserver with Master: d7bef08c4ff046d785bb609f41ad30c5 (127.5.78.2:37253)
I20260926 20:46:19.308130 9564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:45099
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:46:19.338514 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45685
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:19.684238 9891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:19.684573 9891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:19.684775 9891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:19.692939 9891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:19.693073 9891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:19.693270 9891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:19.703734 9891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:19.705240 9891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:19.706733 9891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:19.721136 9896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:19.721105 9899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:19.721200 9897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:19.721922 9891 server_base.cc:1034] running on GCE node
I20260926 20:46:19.722731 9891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:19.723763 9891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:19.725068 9891 hybrid_clock.cc:648] HybridClock initialized: now 1790455579724982 us; error 58 us; skew 500 ppm
I20260926 20:46:19.725462 9891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:19 dist-test-slave-z7j8 krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455579, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:19.728937 9891 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:46:19.730230 9891 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:19.730381 9891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:19.730777 9891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:19.733848 9891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "d87c5120da854608b7b95aa8d7f88e64"
format_stamp: "Formatted at 2026-09-26 20:46:19 on dist-test-slave-z7j8"
server_key: "454af165a0aec1fc09ecf46dc94bdb31"
server_key_iv: "873edc89f054b51029aa6fd8e0cbe84d"
server_key_version: "encryptionkey@0"
I20260926 20:46:19.734784 9891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "d87c5120da854608b7b95aa8d7f88e64"
format_stamp: "Formatted at 2026-09-26 20:46:19 on dist-test-slave-z7j8"
server_key: "454af165a0aec1fc09ecf46dc94bdb31"
server_key_iv: "873edc89f054b51029aa6fd8e0cbe84d"
server_key_version: "encryptionkey@0"
I20260926 20:46:19.740103 9891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:46:19.743865 9905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:19.745443 9891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:19.745667 9891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "d87c5120da854608b7b95aa8d7f88e64"
format_stamp: "Formatted at 2026-09-26 20:46:19 on dist-test-slave-z7j8"
server_key: "454af165a0aec1fc09ecf46dc94bdb31"
server_key_iv: "873edc89f054b51029aa6fd8e0cbe84d"
server_key_version: "encryptionkey@0"
I20260926 20:46:19.745922 9891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:19.762707 9891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:19.767493 9891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:19.767805 9891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:19.768908 9891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:19.770462 9891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:19.770591 9891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:19.770742 9891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:19.770833 9891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:19.792361 9752 heartbeater.cc:503] Master 127.5.78.62:45685 was elected leader, sending a full tablet report...
I20260926 20:46:19.810559 9891 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:40853
I20260926 20:46:19.810606 10017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:40853 every 8 connection(s)
I20260926 20:46:19.811515 9891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:19.819269 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 9891
I20260926 20:46:19.819490 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:19.820008 5432 external_mini_cluster.cc:1428] Setting key 6f60db4f8a84ebd623c6de47e361f11b
Sep 26 20:46:19 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455579, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:19.833271 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.814271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:57381 (local address 127.5.78.62:45685)
0926 20:46:19.814753 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:19.814758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:19.815902 (+ 1144us) server_negotiation.cc:407] Connection header received
0926 20:46:19.817782 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:19.817787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:19.817869 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:19.817976 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:19.819862 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.820908 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.822890 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.823435 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.823575 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.827198 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:19.827204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:19.827207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:19.827247 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:19.829845 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:19.830776 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:19.830780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:19.830783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:19.830918 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:19.831382 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:19.831392 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:19.831394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:19.831669 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:19.831856 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:19.832616 (+ 760us) server_negotiation.cc:299] Negotiation successful
0926 20:46:19.832880 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":265,"threads_started":1}
I20260926 20:46:19.834569 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.814935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:45685 (local address 127.5.78.3:57381)
0926 20:46:19.815707 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:19.815753 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:19.817454 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:19.818217 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:19.818227 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:19.818925 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:19.819635 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.819657 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.821085 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.821091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.822718 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.822731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.823675 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.823682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.823915 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.824562 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:19.824590 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:19.826933 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:19.830025 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:19.830035 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:19.830061 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:19.830626 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:19.831043 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:19.831050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:19.831058 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:19.831249 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:19.831814 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:19.831825 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:19.832211 (+ 386us) client_negotiation.cc:769] Sending connection context
0926 20:46:19.832511 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 20:46:19.833112 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":318,"threads_started":1}
I20260926 20:46:19.836072 10018 heartbeater.cc:347] Connected to a master server at 127.5.78.62:45685
I20260926 20:46:19.836426 10018 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:19.837126 10018 heartbeater.cc:511] Master 127.5.78.62:45685 requested a full tablet report, sending...
I20260926 20:46:19.838729 9564 ts_manager.cc:194] Registered new tserver with Master: d87c5120da854608b7b95aa8d7f88e64 (127.5.78.3:40853)
I20260926 20:46:19.839557 9564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:57381
I20260926 20:46:19.849645 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:46:19.867942 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.854625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.5.78.62:45685)
0926 20:46:19.854840 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:19.854849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:19.855338 (+ 489us) server_negotiation.cc:407] Connection header received
0926 20:46:19.855550 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:19.855560 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:19.855635 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:19.855748 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:19.857248 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.857975 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.859332 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.859776 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.859922 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.862356 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:19.862362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:19.862365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:19.862466 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:19.865116 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:19.865878 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:19.865884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:19.865888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:19.866024 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:19.866431 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:19.866436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:19.866440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:19.866785 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:19.867046 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:19.867371 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 20:46:19.867603 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:46:19.872804 9564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:19.879765 9564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:19.916747 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.893409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.5.78.2:37253)
0926 20:46:19.907007 (+ 13598us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:19.907021 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:19.907127 (+ 106us) server_negotiation.cc:407] Connection header received
0926 20:46:19.907273 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:19.907286 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:19.907634 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:19.907897 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:19.909880 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.911526 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.914170 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.915467 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.915644 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.915877 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:19.916225 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 20:46:19.916398 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13368,"thread_start_us":13262,"threads_started":1}
I20260926 20:46:19.916805 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.893239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37253 (local address 127.0.0.1:34392)
0926 20:46:19.905724 (+ 12485us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:19.905778 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:19.905969 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:19.909179 (+ 3210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:19.909188 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:19.909233 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:19.909659 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.909671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.911760 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.911767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.913826 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.913838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.915717 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.915724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.915943 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.915991 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 20:46:19.916177 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 20:46:19.916390 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12199,"spinlock_wait_cycles":1792,"thread_start_us":15191,"threads_started":1}
I20260926 20:46:19.919013 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.893320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:38993 (local address 127.0.0.1:57178)
0926 20:46:19.908826 (+ 15506us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:19.908852 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:19.908999 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:19.911262 (+ 2263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:19.911270 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:19.911304 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:19.911677 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.911688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.913648 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.913652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.916151 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.916167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.918082 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.918089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.918249 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.918281 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:46:19.918385 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 20:46:19.918550 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15317,"mutex_wait_us":30,"thread_start_us":11974,"threads_started":1}
I20260926 20:46:19.919548 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.894904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:40853 (local address 127.0.0.1:49654)
0926 20:46:19.909238 (+ 14334us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:19.909261 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:19.909459 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:19.910863 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:19.910867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:19.910896 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:19.911393 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.911411 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.913643 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.913648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.916151 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:19.916167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.918611 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.918618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:19.918843 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.918887 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 20:46:19.919009 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 20:46:19.919142 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14165,"mutex_wait_us":28,"spinlock_wait_cycles":4992,"thread_start_us":13975,"threads_started":1}
I20260926 20:46:19.920037 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.895637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.5.78.3:40853)
0926 20:46:19.910139 (+ 14502us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:19.910153 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:19.910197 (+ 44us) server_negotiation.cc:407] Connection header received
0926 20:46:19.910338 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:19.910344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:19.910650 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:19.910884 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:19.911665 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.913460 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.916353 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.917878 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.918013 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.918245 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:19.919562 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0926 20:46:19.919739 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14295,"thread_start_us":14140,"threads_started":1}
I20260926 20:46:19.921698 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:19.893901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.5.78.1:38993)
0926 20:46:19.910639 (+ 16738us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:19.910648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:19.910676 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:46:19.910793 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:19.910800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:19.911037 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:19.911245 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:19.911844 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.913452 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.916353 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:19.917875 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:19.918013 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:19.918256 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:19.918549 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 20:46:19.921389 (+ 2840us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16574,"thread_start_us":16413,"threads_started":1}
I20260926 20:46:19.921502 9820 tablet_service.cc:1467] Processing CreateTablet for tablet 77f593de7ff846199073a06d6e3fe1a5 (DEFAULT_TABLE table=test-table [id=e14c88cb01cc46559db8294495b6c570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:19.923328 9820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f593de7ff846199073a06d6e3fe1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:19.923966 9953 tablet_service.cc:1467] Processing CreateTablet for tablet 77f593de7ff846199073a06d6e3fe1a5 (DEFAULT_TABLE table=test-table [id=e14c88cb01cc46559db8294495b6c570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:19.925458 9953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f593de7ff846199073a06d6e3fe1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:19.925719 9687 tablet_service.cc:1467] Processing CreateTablet for tablet 77f593de7ff846199073a06d6e3fe1a5 (DEFAULT_TABLE table=test-table [id=e14c88cb01cc46559db8294495b6c570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:19.927148 9687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f593de7ff846199073a06d6e3fe1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:19.939739 10037 tablet_bootstrap.cc:492] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Bootstrap starting.
I20260926 20:46:19.943266 10038 tablet_bootstrap.cc:492] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Bootstrap starting.
I20260926 20:46:19.943450 10037 tablet_bootstrap.cc:654] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:19.944847 10037 log.cc:824] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:19.947338 10038 tablet_bootstrap.cc:654] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:19.947561 10039 tablet_bootstrap.cc:492] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Bootstrap starting.
I20260926 20:46:19.948575 10037 tablet_bootstrap.cc:492] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: No bootstrap required, opened a new log
I20260926 20:46:19.948784 10038 log.cc:824] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:19.948940 10037 ts_tablet_manager.cc:1397] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:46:19.951522 10039 tablet_bootstrap.cc:654] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:19.952484 10038 tablet_bootstrap.cc:492] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: No bootstrap required, opened a new log
I20260926 20:46:19.952790 10038 ts_tablet_manager.cc:1397] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 20:46:19.952883 10039 log.cc:824] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:19.956346 10039 tablet_bootstrap.cc:492] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: No bootstrap required, opened a new log
I20260926 20:46:19.956662 10039 ts_tablet_manager.cc:1397] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 20:46:19.958503 10037 raft_consensus.cc:348] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:19.959168 10037 raft_consensus.cc:374] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:19.959319 10037 raft_consensus.cc:729] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87c5120da854608b7b95aa8d7f88e64, State: Initialized, Role: FOLLOWER
I20260926 20:46:19.960034 10037 consensus_queue.cc:260] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:19.961355 10018 heartbeater.cc:503] Master 127.5.78.62:45685 was elected leader, sending a full tablet report...
I20260926 20:46:19.962031 10037 ts_tablet_manager.cc:1428] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260926 20:46:19.964401 10038 raft_consensus.cc:348] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:19.964815 10038 raft_consensus.cc:374] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:19.964948 10038 raft_consensus.cc:729] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7bef08c4ff046d785bb609f41ad30c5, State: Initialized, Role: FOLLOWER
I20260926 20:46:19.965618 10038 consensus_queue.cc:260] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:19.966863 9885 heartbeater.cc:503] Master 127.5.78.62:45685 was elected leader, sending a full tablet report...
I20260926 20:46:19.967629 10038 ts_tablet_manager.cc:1428] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260926 20:46:19.969138 10039 raft_consensus.cc:348] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:19.969712 10039 raft_consensus.cc:374] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:19.969822 10039 raft_consensus.cc:729] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb94252dfe34b9c99cb9ce7b240e2c2, State: Initialized, Role: FOLLOWER
I20260926 20:46:19.970584 10039 consensus_queue.cc:260] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:19.972695 10039 ts_tablet_manager.cc:1428] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260926 20:46:20.012192 9753 tablet.cc:2369] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:46:20.031518 9886 tablet.cc:2369] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:46:20.063334 10019 tablet.cc:2369] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:46:20.113379 10044 raft_consensus.cc:482] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:20.113746 10044 raft_consensus.cc:504] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:20.115667 10044 leader_election.cc:288] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cb94252dfe34b9c99cb9ce7b240e2c2 (127.5.78.1:38993), d87c5120da854608b7b95aa8d7f88e64 (127.5.78.3:40853)
I20260926 20:46:20.122985 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.116288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36523 (local address 127.5.78.1:38993)
0926 20:46:20.116568 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:20.116573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:20.117342 (+ 769us) server_negotiation.cc:407] Connection header received
0926 20:46:20.117581 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:20.117586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:20.117647 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:20.117751 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:20.118344 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.119165 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.121109 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.121904 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.122076 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.122185 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:20.122570 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 20:46:20.122704 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260926 20:46:20.123068 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.116088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:38993 (local address 127.5.78.2:36523)
0926 20:46:20.117211 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:20.117227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:20.117430 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:20.117778 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:20.117782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:20.117807 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:20.118186 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:20.118195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.119302 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.119311 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:20.120922 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:20.120937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.122145 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.122151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:20.122307 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.122339 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:46:20.122536 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 20:46:20.122693 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"thread_start_us":693,"threads_started":1}
I20260926 20:46:20.123267 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.116203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:44363 (local address 127.5.78.3:40853)
0926 20:46:20.116452 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:20.116459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:20.117452 (+ 993us) server_negotiation.cc:407] Connection header received
0926 20:46:20.117655 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:20.117660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:20.117725 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:20.117834 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:20.118587 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.119352 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.121293 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.122160 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.122339 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.122506 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:20.122838 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 20:46:20.122977 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:46:20.123373 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.116136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:40853 (local address 127.5.78.2:44363)
0926 20:46:20.117330 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:20.117348 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:20.117477 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:20.117866 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:20.117871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:20.117914 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:20.118355 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:20.118366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.119514 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.119521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:20.121082 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:20.121096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.122453 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.122460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:20.122655 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.122679 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 20:46:20.122808 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 20:46:20.122952 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"thread_start_us":709,"threads_started":1}
I20260926 20:46:20.124395 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f593de7ff846199073a06d6e3fe1a5" candidate_uuid: "d7bef08c4ff046d785bb609f41ad30c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" is_pre_election: true
I20260926 20:46:20.124400 9973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f593de7ff846199073a06d6e3fe1a5" candidate_uuid: "d7bef08c4ff046d785bb609f41ad30c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d87c5120da854608b7b95aa8d7f88e64" is_pre_election: true
I20260926 20:46:20.124912 9707 raft_consensus.cc:2454] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7bef08c4ff046d785bb609f41ad30c5 in term 0.
I20260926 20:46:20.124931 9973 raft_consensus.cc:2454] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7bef08c4ff046d785bb609f41ad30c5 in term 0.
I20260926 20:46:20.125629 9773 leader_election.cc:302] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb94252dfe34b9c99cb9ce7b240e2c2, d7bef08c4ff046d785bb609f41ad30c5; no voters:
I20260926 20:46:20.126075 10044 raft_consensus.cc:2781] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:20.126224 10044 raft_consensus.cc:482] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:20.126354 10044 raft_consensus.cc:3037] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:20.128625 10044 raft_consensus.cc:504] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:20.129325 10044 leader_election.cc:288] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [CANDIDATE]: Term 1 election: Requested vote from peers 1cb94252dfe34b9c99cb9ce7b240e2c2 (127.5.78.1:38993), d87c5120da854608b7b95aa8d7f88e64 (127.5.78.3:40853)
I20260926 20:46:20.129918 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f593de7ff846199073a06d6e3fe1a5" candidate_uuid: "d7bef08c4ff046d785bb609f41ad30c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2"
I20260926 20:46:20.129932 9973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f593de7ff846199073a06d6e3fe1a5" candidate_uuid: "d7bef08c4ff046d785bb609f41ad30c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d87c5120da854608b7b95aa8d7f88e64"
I20260926 20:46:20.130133 9707 raft_consensus.cc:3037] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:20.130151 9973 raft_consensus.cc:3037] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:20.132377 9973 raft_consensus.cc:2454] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7bef08c4ff046d785bb609f41ad30c5 in term 1.
I20260926 20:46:20.132387 9707 raft_consensus.cc:2454] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7bef08c4ff046d785bb609f41ad30c5 in term 1.
I20260926 20:46:20.132874 9774 leader_election.cc:302] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7bef08c4ff046d785bb609f41ad30c5, d87c5120da854608b7b95aa8d7f88e64; no voters:
I20260926 20:46:20.133201 10044 raft_consensus.cc:2781] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:20.133632 10044 raft_consensus.cc:686] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 1 LEADER]: Becoming Leader. State: Replica: d7bef08c4ff046d785bb609f41ad30c5, State: Running, Role: LEADER
I20260926 20:46:20.134183 10044 consensus_queue.cc:237] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } }
I20260926 20:46:20.139482 9563 catalog_manager.cc:5495] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7bef08c4ff046d785bb609f41ad30c5 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "d7bef08c4ff046d785bb609f41ad30c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bef08c4ff046d785bb609f41ad30c5" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 } health_report { overall_health: UNKNOWN } } }
I20260926 20:46:20.160624 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.154448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.5.78.2:37253)
0926 20:46:20.154632 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:20.154639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:20.154755 (+ 116us) server_negotiation.cc:407] Connection header received
0926 20:46:20.154988 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:20.154992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:20.155080 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:20.155203 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:20.155597 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.156319 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.158244 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.158756 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.158885 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.159476 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:20.160036 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:20.160157 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:20.160284 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 20:46:20.160357 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 20:46:20.170280 9707 raft_consensus.cc:1261] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 1 FOLLOWER]: Refusing update from remote peer d7bef08c4ff046d785bb609f41ad30c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:20.170276 9973 raft_consensus.cc:1261] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 1 FOLLOWER]: Refusing update from remote peer d7bef08c4ff046d785bb609f41ad30c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:20.171339 10048 consensus_queue.cc:1035] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb94252dfe34b9c99cb9ce7b240e2c2" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 38993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:20.171797 10044 consensus_queue.cc:1035] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d87c5120da854608b7b95aa8d7f88e64" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:20.189154 10053 mvcc.cc:205] Tried to move back new op lower bound from 7333706056366579712 to 7333706056233816064. Current Snapshot: MvccSnapshot[applied={T|T < 7333706056366579712}]
I20260926 20:46:20.203310 9563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:20.203608 9563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:20.207624 9563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1e228d578c64e7d8b38c4699aea8d29: Sending DeleteTablet for 3 replicas of tablet 77f593de7ff846199073a06d6e3fe1a5
I20260926 20:46:20.209141 9820 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f593de7ff846199073a06d6e3fe1a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:20 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:34392
I20260926 20:46:20.209151 9953 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f593de7ff846199073a06d6e3fe1a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:20 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:49654
I20260926 20:46:20.210551 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9625
I20260926 20:46:20.210533 9687 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f593de7ff846199073a06d6e3fe1a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:20 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:57178
I20260926 20:46:20.220263 10061 tablet_replica.cc:330] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: stopping tablet replica
I20260926 20:46:20.220980 10061 raft_consensus.cc:2229] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:20.222048 10061 raft_consensus.cc:2258] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:20.224507 10062 tablet_replica.cc:330] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: stopping tablet replica
I20260926 20:46:20.224769 10063 tablet_replica.cc:330] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: stopping tablet replica
I20260926 20:46:20.225026 10062 raft_consensus.cc:2229] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:20.225347 10063 raft_consensus.cc:2229] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:20.225591 10062 raft_consensus.cc:2258] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:20.226452 10063 raft_consensus.cc:2258] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:20.228955 10063 ts_tablet_manager.cc:1905] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:20.229228 10061 ts_tablet_manager.cc:1905] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 20:46:20 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:20.237635 10063 ts_tablet_manager.cc:1918] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:20.237636 10061 ts_tablet_manager.cc:1918] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:20.237864 10061 log.cc:1196] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/77f593de7ff846199073a06d6e3fe1a5
I20260926 20:46:20.238022 10063 log.cc:1196] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/77f593de7ff846199073a06d6e3fe1a5
I20260926 20:46:20.238626 10061 ts_tablet_manager.cc:1939] T 77f593de7ff846199073a06d6e3fe1a5 P d7bef08c4ff046d785bb609f41ad30c5: Deleting consensus metadata
I20260926 20:46:20.238817 10063 ts_tablet_manager.cc:1939] T 77f593de7ff846199073a06d6e3fe1a5 P 1cb94252dfe34b9c99cb9ce7b240e2c2: Deleting consensus metadata
I20260926 20:46:20.240800 9551 catalog_manager.cc:4841] TS d7bef08c4ff046d785bb609f41ad30c5 (127.5.78.2:37253): tablet 77f593de7ff846199073a06d6e3fe1a5 (table test-table [id=e14c88cb01cc46559db8294495b6c570]) successfully deleted
I20260926 20:46:20.241180 9548 catalog_manager.cc:4841] TS 1cb94252dfe34b9c99cb9ce7b240e2c2 (127.5.78.1:38993): tablet 77f593de7ff846199073a06d6e3fe1a5 (table test-table [id=e14c88cb01cc46559db8294495b6c570]) successfully deleted
I20260926 20:46:20.242034 10062 ts_tablet_manager.cc:1905] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:20.244608 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.213238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.5.78.1:38993)
0926 20:46:20.213469 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:20.213475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:20.223688 (+ 10213us) server_negotiation.cc:407] Connection header received
0926 20:46:20.223963 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:20.223969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:20.224079 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:20.224209 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:20.226220 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.227501 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.229874 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.230618 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.230736 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.236498 (+ 5762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:20.236519 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:20.236546 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:20.236607 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:20.240774 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:20.241778 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:20.241786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:20.241788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:20.241943 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:20.242476 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:20.242496 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:20.242506 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:20.242896 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:20.243279 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:20.243816 (+ 537us) server_negotiation.cc:299] Negotiation successful
0926 20:46:20.244041 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:46:20.245401 9747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:20.249498 10062 ts_tablet_manager.cc:1918] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:20.249699 10062 log.cc:1196] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/77f593de7ff846199073a06d6e3fe1a5
I20260926 20:46:20.250257 10062 ts_tablet_manager.cc:1939] T 77f593de7ff846199073a06d6e3fe1a5 P d87c5120da854608b7b95aa8d7f88e64: Deleting consensus metadata
I20260926 20:46:20.252393 9549 catalog_manager.cc:4841] TS d87c5120da854608b7b95aa8d7f88e64 (127.5.78.3:40853): tablet 77f593de7ff846199073a06d6e3fe1a5 (table test-table [id=e14c88cb01cc46559db8294495b6c570]) successfully deleted
I20260926 20:46:20.337452 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9625
I20260926 20:46:20.355082 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9758
Sep 26 20:46:20 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:46:20.373445 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.356004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.5.78.2:37253)
0926 20:46:20.356187 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:20.356194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:20.356217 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:46:20.356353 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:20.356359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:20.356438 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:20.356546 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:20.357809 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.359086 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.360334 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.361060 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.361189 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.365778 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:20.365826 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:20.365854 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:20.365913 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:20.370023 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:20.370941 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:20.370959 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:20.370965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:20.371154 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:20.371612 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:20.371619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:20.371623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:20.372135 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:20.372499 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:20.372784 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 20:46:20.373019 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:46:20.374292 9880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:20.484938 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9758
I20260926 20:46:20.503423 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9891
Sep 26 20:46:20 dist-test-slave-z7j8 krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:46:20.519354 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:20.504197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.5.78.3:40853)
0926 20:46:20.504401 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:20.504409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:20.504448 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:46:20.504650 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:20.504658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:20.504738 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:20.504853 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:20.506108 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.506931 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.508223 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:20.508691 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:20.508822 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:20.513298 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:20.513314 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:20.513331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:20.513372 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:20.516474 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:20.517246 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:20.517255 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:20.517257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:20.517394 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:20.517798 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:20.517802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:20.517805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:20.518163 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:20.518611 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:20.518773 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 20:46:20.519011 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:46:20.520061 10013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:20.607941 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9891
I20260926 20:46:20.624168 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 9533
I20260926 20:46:20.625394 9594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:20.751271 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 9533
2026-09-26T20:46:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4331 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T20:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:20Z Disabled control of system clock
I20260926 20:46:20.821147 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44869
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 20:46:21.161165 10072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:21.161478 10072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:21.161628 10072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:21.168987 10072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:21.169122 10072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:21.169198 10072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:21.169286 10072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:21.169340 10072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:21.178982 10072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:21.180369 10072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:21.182237 10072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:21.194702 10077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:21.194852 10078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:21.194726 10080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:21.195911 10072 server_base.cc:1034] running on GCE node
I20260926 20:46:21.196938 10072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:21.198968 10072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:21.200222 10072 hybrid_clock.cc:648] HybridClock initialized: now 1790455581200159 us; error 69 us; skew 500 ppm
I20260926 20:46:21.200598 10072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:21.203346 10072 webserver.cc:460] Webserver started at http://127.5.78.62:38639/ using document root <none> and password file <none>
I20260926 20:46:21.204141 10072 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:21.204290 10072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:21.204646 10072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:21.207877 10072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "d72a2e3b77a6450d95e736d4eb267b33"
format_stamp: "Formatted at 2026-09-26 20:46:21 on dist-test-slave-z7j8"
server_key: "0a28bf01baa7fe65d8eb978cd2e13582"
server_key_iv: "be11342ec8ba74c565a8a7b0f161d5f5"
server_key_version: "encryptionkey@0"
I20260926 20:46:21.208787 10072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "d72a2e3b77a6450d95e736d4eb267b33"
format_stamp: "Formatted at 2026-09-26 20:46:21 on dist-test-slave-z7j8"
server_key: "0a28bf01baa7fe65d8eb978cd2e13582"
server_key_iv: "be11342ec8ba74c565a8a7b0f161d5f5"
server_key_version: "encryptionkey@0"
I20260926 20:46:21.214560 10072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:46:21.218640 10086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:21.220254 10072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 20:46:21.220496 10072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "d72a2e3b77a6450d95e736d4eb267b33"
format_stamp: "Formatted at 2026-09-26 20:46:21 on dist-test-slave-z7j8"
server_key: "0a28bf01baa7fe65d8eb978cd2e13582"
server_key_iv: "be11342ec8ba74c565a8a7b0f161d5f5"
server_key_version: "encryptionkey@0"
I20260926 20:46:21.220762 10072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:21.227274 10072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:21.228225 10072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:21.228570 10072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:21.248374 10072 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:44869
I20260926 20:46:21.248396 10137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:44869 every 8 connection(s)
I20260926 20:46:21.249820 10072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:21.251423 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10072
I20260926 20:46:21.251622 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:21.252408 5432 external_mini_cluster.cc:1428] Setting key 2002952b908dd44ff2c1bda6f8cb1fa8
I20260926 20:46:21.254967 10138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:21.261008 10141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:21.254587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.5.78.62:44869)
0926 20:46:21.255607 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:21.255619 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:21.255658 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:46:21.257193 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:21.257218 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:21.257640 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:21.258048 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:21.259292 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:21.259298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:21.259314 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:21.259331 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:21.259402 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:21.259417 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:21.259683 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:21.259836 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 20:46:21.260092 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":371,"threads_started":1}
I20260926 20:46:21.266696 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33: Bootstrap starting.
I20260926 20:46:21.270118 10138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:21.271198 10138 log.cc:824] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:21.274235 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33: No bootstrap required, opened a new log
I20260926 20:46:21.281916 10138 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72a2e3b77a6450d95e736d4eb267b33" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44869 } }
I20260926 20:46:21.282306 10138 raft_consensus.cc:374] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:21.282384 10138 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72a2e3b77a6450d95e736d4eb267b33, State: Initialized, Role: FOLLOWER
I20260926 20:46:21.283023 10138 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72a2e3b77a6450d95e736d4eb267b33" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44869 } }
I20260926 20:46:21.283226 10138 raft_consensus.cc:388] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:21.283313 10138 raft_consensus.cc:482] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:21.283445 10138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:21.285534 10138 raft_consensus.cc:504] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72a2e3b77a6450d95e736d4eb267b33" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44869 } }
I20260926 20:46:21.285991 10138 leader_election.cc:302] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d72a2e3b77a6450d95e736d4eb267b33; no voters:
I20260926 20:46:21.286598 10138 leader_election.cc:288] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:21.286798 10143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:21.287567 10143 raft_consensus.cc:686] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [term 1 LEADER]: Becoming Leader. State: Replica: d72a2e3b77a6450d95e736d4eb267b33, State: Running, Role: LEADER
I20260926 20:46:21.288203 10143 consensus_queue.cc:237] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72a2e3b77a6450d95e736d4eb267b33" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44869 } }
I20260926 20:46:21.288959 10138 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:21.291613 10145 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d72a2e3b77a6450d95e736d4eb267b33. Latest consensus state: current_term: 1 leader_uuid: "d72a2e3b77a6450d95e736d4eb267b33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72a2e3b77a6450d95e736d4eb267b33" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44869 } } }
I20260926 20:46:21.291651 10144 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d72a2e3b77a6450d95e736d4eb267b33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d72a2e3b77a6450d95e736d4eb267b33" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44869 } } }
I20260926 20:46:21.292043 10145 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:21.292078 10144 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:21.294430 10151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:21.299381 10151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:21.308458 10151 catalog_manager.cc:1347] Generated new cluster ID: 42a7d8ed427348bfa1a4a364ca6776aa
I20260926 20:46:21.308616 10151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:21.326475 10151 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:21.326920 10151 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:21.345443 10151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33: Generated new TSK 0
I20260926 20:46:21.346494 10151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:46:21.365921 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44869
--builtin_ntp_servers=127.5.78.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 20:46:21.736243 10162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:21.736619 10162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:21.736801 10162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:21.744470 10162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:21.744663 10162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:21.744856 10162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:21.754645 10162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:21.756062 10162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:21.758008 10162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:21.773169 10167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:21.773154 10170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:21.773123 10168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:21.774144 10162 server_base.cc:1034] running on GCE node
I20260926 20:46:21.774914 10162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:21.776065 10162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:21.777315 10162 hybrid_clock.cc:648] HybridClock initialized: now 1790455581777276 us; error 42 us; skew 500 ppm
I20260926 20:46:21.777673 10162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:21.780218 10162 webserver.cc:460] Webserver started at http://127.5.78.1:36761/ using document root <none> and password file <none>
I20260926 20:46:21.780961 10162 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:21.781109 10162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:21.781437 10162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:21.784565 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "4b508d0f6d7d424baac2bc5b9c6de426"
format_stamp: "Formatted at 2026-09-26 20:46:21 on dist-test-slave-z7j8"
server_key: "e779e5e7ecaff718b1dca156e35e092e"
server_key_iv: "8fdc03275607db52d1d87c36f3a4d815"
server_key_version: "encryptionkey@0"
I20260926 20:46:21.785437 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "4b508d0f6d7d424baac2bc5b9c6de426"
format_stamp: "Formatted at 2026-09-26 20:46:21 on dist-test-slave-z7j8"
server_key: "e779e5e7ecaff718b1dca156e35e092e"
server_key_iv: "8fdc03275607db52d1d87c36f3a4d815"
server_key_version: "encryptionkey@0"
I20260926 20:46:21.791189 10162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:46:21.795217 10176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:21.796777 10162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:21.797012 10162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "4b508d0f6d7d424baac2bc5b9c6de426"
format_stamp: "Formatted at 2026-09-26 20:46:21 on dist-test-slave-z7j8"
server_key: "e779e5e7ecaff718b1dca156e35e092e"
server_key_iv: "8fdc03275607db52d1d87c36f3a4d815"
server_key_version: "encryptionkey@0"
I20260926 20:46:21.797283 10162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:21.803447 10162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:21.804358 10162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:21.804711 10162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:21.805727 10162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:21.807323 10162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:21.807448 10162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:21.807598 10162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:21.807689 10162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:21.841717 10162 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:34981
I20260926 20:46:21.841769 10288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:34981 every 8 connection(s)
I20260926 20:46:21.843147 10162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:21.849803 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10162
I20260926 20:46:21.849952 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:21.850611 5432 external_mini_cluster.cc:1428] Setting key cd53cfcdc685dd329bf68b7cc9742304
I20260926 20:46:21.851420 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:21.845771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:40575 (local address 127.5.78.62:44869)
0926 20:46:21.846168 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:21.846172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:21.847509 (+ 1337us) server_negotiation.cc:407] Connection header received
0926 20:46:21.849144 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:21.849148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:21.849197 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:21.849293 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:21.850723 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:21.850729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:21.850733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:21.850753 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:21.850806 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:21.850820 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:21.850946 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:21.851106 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 20:46:21.851166 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":200,"threads_started":1}
I20260926 20:46:21.852422 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:21.846344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44869 (local address 127.5.78.1:40575)
0926 20:46:21.847351 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:21.847394 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:21.848891 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:21.849497 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:21.849505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:21.849598 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:21.850363 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:21.850365 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:21.850449 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:21.850552 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:21.850910 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:21.850918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:21.850923 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 20:46:21.851136 (+ 213us) client_negotiation.cc:240] Negotiation successful
0926 20:46:21.851389 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":291,"threads_started":1}
I20260926 20:46:21.853610 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44869
--builtin_ntp_servers=127.5.78.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 20:46:21.853986 10289 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44869
I20260926 20:46:21.854310 10289 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:21.855064 10289 heartbeater.cc:511] Master 127.5.78.62:44869 requested a full tablet report, sending...
I20260926 20:46:21.857443 10103 ts_manager.cc:194] Registered new tserver with Master: 4b508d0f6d7d424baac2bc5b9c6de426 (127.5.78.1:34981)
W20260926 20:46:22.183622 10293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:22.183928 10293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:22.184046 10293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:22.191499 10293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:22.191622 10293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:22.191735 10293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:22.201438 10293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:22.202934 10293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:22.204799 10293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:22.219496 10301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:22.219522 10299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:22.219592 10298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:22.220507 10293 server_base.cc:1034] running on GCE node
I20260926 20:46:22.221132 10293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:22.222103 10293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:22.223323 10293 hybrid_clock.cc:648] HybridClock initialized: now 1790455582223241 us; error 69 us; skew 500 ppm
I20260926 20:46:22.223707 10293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:22.226552 10293 webserver.cc:460] Webserver started at http://127.5.78.2:46377/ using document root <none> and password file <none>
I20260926 20:46:22.227326 10293 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:22.227429 10293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:22.227828 10293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:22.231242 10293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "80d1395a931d4ac3867ec96c70eee609"
format_stamp: "Formatted at 2026-09-26 20:46:22 on dist-test-slave-z7j8"
server_key: "c66be789e83d73a5486124c6c9bcff6b"
server_key_iv: "a750d6cf322aa8827dcb8e209033f014"
server_key_version: "encryptionkey@0"
I20260926 20:46:22.232187 10293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "80d1395a931d4ac3867ec96c70eee609"
format_stamp: "Formatted at 2026-09-26 20:46:22 on dist-test-slave-z7j8"
server_key: "c66be789e83d73a5486124c6c9bcff6b"
server_key_iv: "a750d6cf322aa8827dcb8e209033f014"
server_key_version: "encryptionkey@0"
I20260926 20:46:22.238507 10293 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260926 20:46:22.242184 10307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:22.243769 10293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 20:46:22.243963 10293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "80d1395a931d4ac3867ec96c70eee609"
format_stamp: "Formatted at 2026-09-26 20:46:22 on dist-test-slave-z7j8"
server_key: "c66be789e83d73a5486124c6c9bcff6b"
server_key_iv: "a750d6cf322aa8827dcb8e209033f014"
server_key_version: "encryptionkey@0"
I20260926 20:46:22.244212 10293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:22.250273 10293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:22.251196 10293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:22.251478 10293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:22.252452 10293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:22.254027 10293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:22.254170 10293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:22.254302 10293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:22.254402 10293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:22.288967 10293 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:37713
I20260926 20:46:22.288985 10419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:37713 every 8 connection(s)
I20260926 20:46:22.290331 10293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:22.295235 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10293
I20260926 20:46:22.295389 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:22.295804 5432 external_mini_cluster.cc:1428] Setting key ec41cda3c217598f624b0eece396d541
I20260926 20:46:22.298779 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44869
--builtin_ntp_servers=127.5.78.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 20:46:22.299816 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.293122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:40115 (local address 127.5.78.62:44869)
0926 20:46:22.293341 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.293346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.295084 (+ 1738us) server_negotiation.cc:407] Connection header received
0926 20:46:22.297220 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.297224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.297274 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.297350 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.298982 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.298990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.298998 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.299021 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.299063 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.299080 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.299208 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.299390 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.299457 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:46:22.300856 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.293739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44869 (local address 127.5.78.2:40115)
0926 20:46:22.294908 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.294950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.296881 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.297697 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.297709 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.297852 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.298631 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.298633 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.298677 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.298770 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.299192 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.299200 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.299206 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.299429 (+ 223us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.299656 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":332,"threads_started":1}
I20260926 20:46:22.302361 10420 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44869
I20260926 20:46:22.302733 10420 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:22.303356 10420 heartbeater.cc:511] Master 127.5.78.62:44869 requested a full tablet report, sending...
I20260926 20:46:22.304804 10103 ts_manager.cc:194] Registered new tserver with Master: 80d1395a931d4ac3867ec96c70eee609 (127.5.78.2:37713)
W20260926 20:46:22.645206 10423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:22.645531 10423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:22.645710 10423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:22.653426 10423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:22.653551 10423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:22.653753 10423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:22.663218 10423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:22.664551 10423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:22.666464 10423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:22.680665 10428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:22.680708 10429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:22.680758 10431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:22.683930 10423 server_base.cc:1034] running on GCE node
I20260926 20:46:22.684602 10423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:22.685571 10423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:22.686786 10423 hybrid_clock.cc:648] HybridClock initialized: now 1790455582686739 us; error 38 us; skew 500 ppm
I20260926 20:46:22.687175 10423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:22.689766 10423 webserver.cc:460] Webserver started at http://127.5.78.3:40479/ using document root <none> and password file <none>
I20260926 20:46:22.690598 10423 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:22.690742 10423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:22.691080 10423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:22.694028 10423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "8c7f227bad854f72a3670932f0ebb478"
format_stamp: "Formatted at 2026-09-26 20:46:22 on dist-test-slave-z7j8"
server_key: "3ad34ac630d1f43512209241b9fe9ea4"
server_key_iv: "0756280207effb08a96a62bcc73c2504"
server_key_version: "encryptionkey@0"
I20260926 20:46:22.694820 10423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "8c7f227bad854f72a3670932f0ebb478"
format_stamp: "Formatted at 2026-09-26 20:46:22 on dist-test-slave-z7j8"
server_key: "3ad34ac630d1f43512209241b9fe9ea4"
server_key_iv: "0756280207effb08a96a62bcc73c2504"
server_key_version: "encryptionkey@0"
I20260926 20:46:22.699831 10423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 20:46:22.703279 10437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:22.704551 10423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:22.704716 10423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "8c7f227bad854f72a3670932f0ebb478"
format_stamp: "Formatted at 2026-09-26 20:46:22 on dist-test-slave-z7j8"
server_key: "3ad34ac630d1f43512209241b9fe9ea4"
server_key_iv: "0756280207effb08a96a62bcc73c2504"
server_key_version: "encryptionkey@0"
I20260926 20:46:22.704937 10423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:22.710465 10423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:22.711266 10423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:22.711540 10423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:22.712471 10423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:22.713855 10423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:22.713939 10423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:22.714090 10423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:22.714190 10423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:22.747402 10423 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:44173
I20260926 20:46:22.747459 10549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:44173 every 8 connection(s)
I20260926 20:46:22.748809 10423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:22.750506 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10423
I20260926 20:46:22.750674 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:22.751086 5432 external_mini_cluster.cc:1428] Setting key 10f960ec1afbde1f380ab86b93d4b48e
I20260926 20:46:22.758742 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.751619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:60367 (local address 127.5.78.62:44869)
0926 20:46:22.751837 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.751845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.754579 (+ 2734us) server_negotiation.cc:407] Connection header received
0926 20:46:22.756307 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.756316 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.756387 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.756496 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.757889 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.757896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.757900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.757922 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.757965 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.757980 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.758108 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.758295 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.758363 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:46:22.759545 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.752320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44869 (local address 127.5.78.3:60367)
0926 20:46:22.754360 (+ 2040us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.754448 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.756034 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.756725 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.756734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.756822 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.757576 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.757580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.757633 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.757720 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.758108 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.758115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.758121 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.758322 (+ 201us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.758577 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1450,"thread_start_us":532,"threads_started":1}
I20260926 20:46:22.760861 10550 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44869
I20260926 20:46:22.761193 10550 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:22.761793 10550 heartbeater.cc:511] Master 127.5.78.62:44869 requested a full tablet report, sending...
I20260926 20:46:22.763162 10103 ts_manager.cc:194] Registered new tserver with Master: 8c7f227bad854f72a3670932f0ebb478 (127.5.78.3:44173)
I20260926 20:46:22.766630 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:46:22.773566 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.770989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.5.78.62:44869)
0926 20:46:22.771191 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.771195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.771645 (+ 450us) server_negotiation.cc:407] Connection header received
0926 20:46:22.771771 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.771776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.771841 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.771931 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.773011 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.773015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.773018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.773033 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.773065 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.773083 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.773185 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.773312 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.773359 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 20:46:22.777667 10103 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:22.784703 10103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:22.802114 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.796902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:44173 (local address 127.0.0.1:52050)
0926 20:46:22.798295 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.798329 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.798508 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.799156 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.799167 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.799299 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.800164 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.800167 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.800198 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.800292 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.801138 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.801153 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.801165 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.801408 (+ 243us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.801512 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"thread_start_us":1050,"threads_started":1}
I20260926 20:46:22.802248 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.797048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52050 (local address 127.5.78.3:44173)
0926 20:46:22.797699 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.797708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.798458 (+ 750us) server_negotiation.cc:407] Connection header received
0926 20:46:22.798655 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.798666 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.798925 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.799783 (+ 858us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.800872 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.800880 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.800897 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.800914 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.800973 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.800991 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.801196 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.801685 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.801801 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":316,"threads_started":1}
I20260926 20:46:22.807298 10485 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0321457ba43b19bbdfcef64e099ef (DEFAULT_TABLE table=test-table [id=4f4099a7bb7343a68b1026e5c5cf023b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:22.809173 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.799544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37713 (local address 127.0.0.1:59588)
0926 20:46:22.802384 (+ 2840us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.804109 (+ 1725us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.804210 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.804951 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.804959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.805014 (+ 55us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.805606 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.805612 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.805634 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.805690 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.808564 (+ 2874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.808573 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.808582 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.808700 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.808773 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2718,"threads_started":1}
I20260926 20:46:22.809518 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0321457ba43b19bbdfcef64e099ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:22.810225 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.800502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.5.78.2:37713)
0926 20:46:22.801009 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.801018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.804352 (+ 3334us) server_negotiation.cc:407] Connection header received
0926 20:46:22.804444 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.804455 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.804745 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.805338 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.808203 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.808215 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.808233 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.808265 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.808331 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.808410 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.809531 (+ 1121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.809664 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.809789 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260926 20:46:22.813866 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.797839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.5.78.1:34981)
0926 20:46:22.798989 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.798999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.799553 (+ 554us) server_negotiation.cc:407] Connection header received
0926 20:46:22.799801 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.799809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.800029 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.800216 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.802830 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.802841 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.802858 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.809189 (+ 6331us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.809278 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.809308 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.809431 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.813392 (+ 3961us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.813505 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1006,"thread_start_us":883,"threads_started":1}
I20260926 20:46:22.813766 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.797215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:53148)
0926 20:46:22.799437 (+ 2222us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.799461 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.799646 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.801121 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.801126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.801151 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.802543 (+ 1392us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.802549 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.802570 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.802624 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.813193 (+ 10569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.813198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.813202 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.813346 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.813403 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2065,"thread_start_us":1927,"threads_started":1}
I20260926 20:46:22.815935 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0321457ba43b19bbdfcef64e099ef (DEFAULT_TABLE table=test-table [id=4f4099a7bb7343a68b1026e5c5cf023b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:22.817368 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0321457ba43b19bbdfcef64e099ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:22.818184 10224 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0321457ba43b19bbdfcef64e099ef (DEFAULT_TABLE table=test-table [id=4f4099a7bb7343a68b1026e5c5cf023b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:22.819585 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0321457ba43b19bbdfcef64e099ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:22.824200 10568 tablet_bootstrap.cc:492] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Bootstrap starting.
I20260926 20:46:22.828142 10568 tablet_bootstrap.cc:654] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:22.829391 10568 log.cc:824] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:22.830963 10569 tablet_bootstrap.cc:492] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Bootstrap starting.
I20260926 20:46:22.832474 10571 tablet_bootstrap.cc:492] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: Bootstrap starting.
I20260926 20:46:22.833964 10568 tablet_bootstrap.cc:492] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: No bootstrap required, opened a new log
I20260926 20:46:22.834276 10568 ts_tablet_manager.cc:1397] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260926 20:46:22.834372 10569 tablet_bootstrap.cc:654] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:22.835425 10569 log.cc:824] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:22.836128 10571 tablet_bootstrap.cc:654] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:22.837322 10571 log.cc:824] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:22.838336 10569 tablet_bootstrap.cc:492] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: No bootstrap required, opened a new log
I20260926 20:46:22.838682 10569 ts_tablet_manager.cc:1397] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:46:22.840521 10571 tablet_bootstrap.cc:492] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: No bootstrap required, opened a new log
I20260926 20:46:22.840793 10571 ts_tablet_manager.cc:1397] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:46:22.843993 10568 raft_consensus.cc:348] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.844455 10568 raft_consensus.cc:374] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:22.844581 10568 raft_consensus.cc:729] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7f227bad854f72a3670932f0ebb478, State: Initialized, Role: FOLLOWER
I20260926 20:46:22.845252 10568 consensus_queue.cc:260] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.846987 10568 ts_tablet_manager.cc:1428] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 20:46:22.847579 10550 heartbeater.cc:503] Master 127.5.78.62:44869 was elected leader, sending a full tablet report...
I20260926 20:46:22.848670 10569 raft_consensus.cc:348] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.849123 10569 raft_consensus.cc:374] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:22.849229 10569 raft_consensus.cc:729] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d1395a931d4ac3867ec96c70eee609, State: Initialized, Role: FOLLOWER
I20260926 20:46:22.849939 10569 consensus_queue.cc:260] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.853127 10571 raft_consensus.cc:348] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.853681 10571 raft_consensus.cc:374] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:22.853829 10571 raft_consensus.cc:729] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b508d0f6d7d424baac2bc5b9c6de426, State: Initialized, Role: FOLLOWER
I20260926 20:46:22.854709 10571 consensus_queue.cc:260] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.856488 10420 heartbeater.cc:503] Master 127.5.78.62:44869 was elected leader, sending a full tablet report...
I20260926 20:46:22.856967 10569 ts_tablet_manager.cc:1428] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Time spent starting tablet: real 0.018s user 0.011s sys 0.006s
I20260926 20:46:22.857883 10289 heartbeater.cc:503] Master 127.5.78.62:44869 was elected leader, sending a full tablet report...
I20260926 20:46:22.858511 10571 ts_tablet_manager.cc:1428] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260926 20:46:22.924997 10574 raft_consensus.cc:482] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:22.925379 10574 raft_consensus.cc:504] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.927328 10574 leader_election.cc:288] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b508d0f6d7d424baac2bc5b9c6de426 (127.5.78.1:34981), 80d1395a931d4ac3867ec96c70eee609 (127.5.78.2:37713)
I20260926 20:46:22.930322 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.927727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:59929 (local address 127.5.78.1:34981)
0926 20:46:22.927912 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.927920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.928009 (+ 89us) server_negotiation.cc:407] Connection header received
0926 20:46:22.928260 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.928268 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.928340 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.928464 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.929645 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.929650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.929654 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.929669 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.929701 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.929747 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.929881 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.930021 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.930089 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:46:22.930377 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.927664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.5.78.3:59929)
0926 20:46:22.927902 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.927925 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.928068 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.928591 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.928603 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.928627 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.929371 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.929377 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.929404 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.929471 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.929864 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.929867 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.929871 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.930008 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.930092 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 20:46:22.931263 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.928251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:47897 (local address 127.5.78.2:37713)
0926 20:46:22.928465 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.928473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.928950 (+ 477us) server_negotiation.cc:407] Connection header received
0926 20:46:22.929059 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.929065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.929124 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.929228 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.930372 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.930379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.930382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.930422 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.930459 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.930475 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.930592 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.930896 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.930945 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:46:22.931313 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.928105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37713 (local address 127.5.78.3:47897)
0926 20:46:22.928822 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:22.928839 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:22.928950 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:22.929303 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:22.929314 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:22.929341 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:22.930148 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:46:22.930151 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:22.930167 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:46:22.930207 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:22.930726 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:22.930732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:22.930739 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:46:22.930881 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 20:46:22.930934 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":398,"threads_started":1}
I20260926 20:46:22.931565 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0321457ba43b19bbdfcef64e099ef" candidate_uuid: "8c7f227bad854f72a3670932f0ebb478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" is_pre_election: true
I20260926 20:46:22.932155 10244 raft_consensus.cc:2454] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c7f227bad854f72a3670932f0ebb478 in term 0.
I20260926 20:46:22.932205 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0321457ba43b19bbdfcef64e099ef" candidate_uuid: "8c7f227bad854f72a3670932f0ebb478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80d1395a931d4ac3867ec96c70eee609" is_pre_election: true
I20260926 20:46:22.932729 10375 raft_consensus.cc:2454] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c7f227bad854f72a3670932f0ebb478 in term 0.
I20260926 20:46:22.932817 10440 leader_election.cc:302] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b508d0f6d7d424baac2bc5b9c6de426, 8c7f227bad854f72a3670932f0ebb478; no voters:
I20260926 20:46:22.933266 10574 raft_consensus.cc:2781] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:22.933424 10574 raft_consensus.cc:482] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:22.933547 10574 raft_consensus.cc:3037] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:22.935882 10574 raft_consensus.cc:504] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.936755 10574 leader_election.cc:288] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [CANDIDATE]: Term 1 election: Requested vote from peers 4b508d0f6d7d424baac2bc5b9c6de426 (127.5.78.1:34981), 80d1395a931d4ac3867ec96c70eee609 (127.5.78.2:37713)
I20260926 20:46:22.937284 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0321457ba43b19bbdfcef64e099ef" candidate_uuid: "8c7f227bad854f72a3670932f0ebb478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80d1395a931d4ac3867ec96c70eee609"
I20260926 20:46:22.937270 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0321457ba43b19bbdfcef64e099ef" candidate_uuid: "8c7f227bad854f72a3670932f0ebb478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b508d0f6d7d424baac2bc5b9c6de426"
I20260926 20:46:22.937490 10244 raft_consensus.cc:3037] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:22.937490 10375 raft_consensus.cc:3037] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:22.940876 10375 raft_consensus.cc:2454] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c7f227bad854f72a3670932f0ebb478 in term 1.
I20260926 20:46:22.940876 10244 raft_consensus.cc:2454] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c7f227bad854f72a3670932f0ebb478 in term 1.
I20260926 20:46:22.941282 10440 leader_election.cc:302] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b508d0f6d7d424baac2bc5b9c6de426, 8c7f227bad854f72a3670932f0ebb478; no voters:
I20260926 20:46:22.941555 10574 raft_consensus.cc:2781] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:22.941967 10574 raft_consensus.cc:686] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 1 LEADER]: Becoming Leader. State: Replica: 8c7f227bad854f72a3670932f0ebb478, State: Running, Role: LEADER
I20260926 20:46:22.942512 10574 consensus_queue.cc:237] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } }
I20260926 20:46:22.947142 10103 catalog_manager.cc:5495] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c7f227bad854f72a3670932f0ebb478 (127.5.78.3). New cstate: current_term: 1 leader_uuid: "8c7f227bad854f72a3670932f0ebb478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7f227bad854f72a3670932f0ebb478" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 } health_report { overall_health: UNKNOWN } } }
I20260926 20:46:22.965466 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:22.963401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.5.78.3:44173)
0926 20:46:22.963609 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:22.963615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:22.963647 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:46:22.963733 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:22.963736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:22.963813 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:22.963929 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:22.964852 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:22.964860 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:22.964869 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:22.964883 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:22.964913 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:22.964923 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:22.965052 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:22.965163 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 20:46:22.965220 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:46:22.973948 10244 raft_consensus.cc:1261] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 1 FOLLOWER]: Refusing update from remote peer 8c7f227bad854f72a3670932f0ebb478: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:22.973989 10375 raft_consensus.cc:1261] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 1 FOLLOWER]: Refusing update from remote peer 8c7f227bad854f72a3670932f0ebb478: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:22.974960 10574 consensus_queue.cc:1035] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b508d0f6d7d424baac2bc5b9c6de426" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:22.975345 10578 consensus_queue.cc:1035] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80d1395a931d4ac3867ec96c70eee609" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:22.986125 10581 mvcc.cc:205] Tried to move back new op lower bound from 7333706067851100160 to 7333706067736629248. Current Snapshot: MvccSnapshot[applied={T|T < 7333706067851100160}]
I20260926 20:46:22.988806 10582 mvcc.cc:205] Tried to move back new op lower bound from 7333706067851100160 to 7333706067736629248. Current Snapshot: MvccSnapshot[applied={T|T < 7333706067851100160}]
I20260926 20:46:22.993244 10583 mvcc.cc:205] Tried to move back new op lower bound from 7333706067851100160 to 7333706067736629248. Current Snapshot: MvccSnapshot[applied={T|T < 7333706067851100160}]
I20260926 20:46:23.003742 10103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:23.004015 10103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:23.007892 10103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d72a2e3b77a6450d95e736d4eb267b33: Sending DeleteTablet for 3 replicas of tablet 06c0321457ba43b19bbdfcef64e099ef
I20260926 20:46:23.009212 10485 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0321457ba43b19bbdfcef64e099ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:23 UTC) from {username='slave'} at 127.0.0.1:52050
I20260926 20:46:23.009200 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0321457ba43b19bbdfcef64e099ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:23 UTC) from {username='slave'} at 127.0.0.1:53148
I20260926 20:46:23.010614 10355 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0321457ba43b19bbdfcef64e099ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:23 UTC) from {username='slave'} at 127.0.0.1:59588
I20260926 20:46:23.010771 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10162
I20260926 20:46:23.011272 10591 tablet_replica.cc:330] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: stopping tablet replica
I20260926 20:46:23.018786 10591 raft_consensus.cc:2229] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:23.018919 10592 tablet_replica.cc:330] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: stopping tablet replica
I20260926 20:46:23.019543 10592 raft_consensus.cc:2229] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:23.019585 10591 raft_consensus.cc:2258] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:23.020289 10592 raft_consensus.cc:2258] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:23.021508 10593 tablet_replica.cc:330] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: stopping tablet replica
I20260926 20:46:23.022120 10593 raft_consensus.cc:2229] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:23.022749 10593 raft_consensus.cc:2258] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:23.024269 10591 ts_tablet_manager.cc:1905] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:23.024665 10592 ts_tablet_manager.cc:1905] T 06c0321457ba43b19bbdfcef64e099ef P 4b508d0f6d7d424baac2bc5b9c6de426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:23.025609 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.012607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.5.78.1:34981)
0926 20:46:23.012765 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:23.012769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:23.021001 (+ 8232us) server_negotiation.cc:407] Connection header received
0926 20:46:23.021738 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:23.021745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:23.021860 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:23.021971 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:23.023510 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:23.023517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:23.023523 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:23.023552 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:23.023596 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:23.023614 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:23.023767 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:23.025217 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0926 20:46:23.025290 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 20:46:23.027181 10284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:23.027428 10593 ts_tablet_manager.cc:1905] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:23.034087 10591 ts_tablet_manager.cc:1918] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:23.034500 10591 log.cc:1196] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/06c0321457ba43b19bbdfcef64e099ef
I20260926 20:46:23.035089 10593 ts_tablet_manager.cc:1918] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:23.035332 10593 log.cc:1196] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/06c0321457ba43b19bbdfcef64e099ef
I20260926 20:46:23.035379 10591 ts_tablet_manager.cc:1939] T 06c0321457ba43b19bbdfcef64e099ef P 8c7f227bad854f72a3670932f0ebb478: Deleting consensus metadata
I20260926 20:46:23.035946 10593 ts_tablet_manager.cc:1939] T 06c0321457ba43b19bbdfcef64e099ef P 80d1395a931d4ac3867ec96c70eee609: Deleting consensus metadata
I20260926 20:46:23.037930 10089 catalog_manager.cc:4841] TS 80d1395a931d4ac3867ec96c70eee609 (127.5.78.2:37713): tablet 06c0321457ba43b19bbdfcef64e099ef (table test-table [id=4f4099a7bb7343a68b1026e5c5cf023b]) successfully deleted
I20260926 20:46:23.037961 10087 catalog_manager.cc:4841] TS 8c7f227bad854f72a3670932f0ebb478 (127.5.78.3:44173): tablet 06c0321457ba43b19bbdfcef64e099ef (table test-table [id=4f4099a7bb7343a68b1026e5c5cf023b]) successfully deleted
I20260926 20:46:23.143826 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10162
I20260926 20:46:23.157878 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10293
W20260926 20:46:23.157931 10089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.78.1:34981 (error 108)
I20260926 20:46:23.159972 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.159422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:53172)
0926 20:46:23.159666 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.159797 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260926 20:46:23.160292 10089 catalog_manager.cc:4584] TS 4b508d0f6d7d424baac2bc5b9c6de426 (127.5.78.1:34981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06c0321457ba43b19bbdfcef64e099ef: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
I20260926 20:46:23.161558 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.158795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.5.78.2:37713)
0926 20:46:23.159001 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:23.159009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:23.159480 (+ 471us) server_negotiation.cc:407] Connection header received
0926 20:46:23.159715 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:23.159722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:23.159834 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:23.159943 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:23.160915 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:23.160921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:23.160927 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:23.160948 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:23.160987 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:23.161004 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:23.161154 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:23.161201 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 20:46:23.161261 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:46:23.162317 10415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:23.213973 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.213407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:53182)
0926 20:46:23.213664 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.213775 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260926 20:46:23.215147 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.214747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:53194)
0926 20:46:23.214905 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.215001 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 20:46:23.252522 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10293
I20260926 20:46:23.269511 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10423
I20260926 20:46:23.272647 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.270252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.5.78.3:44173)
0926 20:46:23.270474 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:23.270480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:23.270563 (+ 83us) server_negotiation.cc:407] Connection header received
0926 20:46:23.270839 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:23.270843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:23.270896 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:23.271008 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:23.271929 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:23.271934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:23.271939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:23.271957 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:23.271996 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:23.272008 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:23.272153 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:23.272259 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 20:46:23.272321 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:46:23.273336 10545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:23.276335 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.275852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:33868)
0926 20:46:23.276050 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.276153 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 20:46:23.277284 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.276903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:33880)
0926 20:46:23.277055 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.277136 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 20:46:23.357926 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.357438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:33886)
0926 20:46:23.357666 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.357754 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 20:46:23.359032 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:23.358598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34981 (local address 127.0.0.1:33896)
0926 20:46:23.358823 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:23.358920 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 20:46:23.387136 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10423
I20260926 20:46:23.406488 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10072
I20260926 20:46:23.407716 10133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:23.511302 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10072
2026-09-26T20:46:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2758 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 20:46:23.552630 5432 mini_oidc.cc:152] Starting JWKS server
I20260926 20:46:23.555706 5432 webserver.cc:460] Webserver started at https://127.0.0.1:42073/ using document root <none> and password file <none>
I20260926 20:46:23.556106 5432 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 20:46:23.556913 5432 webserver.cc:460] Webserver started at http://127.0.0.1:40687/ using document root <none> and password file <none>
2026-09-26T20:46:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:23Z Disabled control of system clock
I20260926 20:46:23.585803 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:35731
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:43869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:35731
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42073/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/testchainca.cert with env {}
W20260926 20:46:23.916369 10605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:23.916759 10605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:23.916930 10605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:23.924350 10605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:23.924516 10605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:23.924615 10605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:23.924726 10605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:23.924816 10605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:46:23.924906 10605 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:46:23.924980 10605 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42073/jwks.json
I20260926 20:46:23.935043 10605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:43869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:35731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:35731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42073/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:23.936450 10605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:23.937855 10605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:23.951010 10610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:23.951085 10611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:23.951033 10613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:23.951891 10605 server_base.cc:1034] running on GCE node
I20260926 20:46:23.952826 10605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:23.954391 10605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:23.955718 10605 hybrid_clock.cc:648] HybridClock initialized: now 1790455583955660 us; error 44 us; skew 500 ppm
I20260926 20:46:23.956197 10605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:23.959137 10605 webserver.cc:460] Webserver started at http://127.5.78.62:45699/ using document root <none> and password file <none>
I20260926 20:46:23.959964 10605 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:23.960078 10605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:23.960451 10605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:23.963831 10605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8"
format_stamp: "Formatted at 2026-09-26 20:46:23 on dist-test-slave-z7j8"
server_key: "a99ba8c19fb29ec92eb69d799fad55a9"
server_key_iv: "9cca262b46a8468357aedaeadeb33253"
server_key_version: "encryptionkey@0"
I20260926 20:46:23.964825 10605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8"
format_stamp: "Formatted at 2026-09-26 20:46:23 on dist-test-slave-z7j8"
server_key: "a99ba8c19fb29ec92eb69d799fad55a9"
server_key_iv: "9cca262b46a8468357aedaeadeb33253"
server_key_version: "encryptionkey@0"
I20260926 20:46:23.970781 10605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:46:23.974867 10619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:23.976477 10605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:23.976712 10605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8"
format_stamp: "Formatted at 2026-09-26 20:46:23 on dist-test-slave-z7j8"
server_key: "a99ba8c19fb29ec92eb69d799fad55a9"
server_key_iv: "9cca262b46a8468357aedaeadeb33253"
server_key_version: "encryptionkey@0"
I20260926 20:46:23.976981 10605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:23.992059 10605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:23.992954 10605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:23.993256 10605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:24.012830 10605 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:35731
I20260926 20:46:24.012867 10670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:35731 every 8 connection(s)
I20260926 20:46:24.014230 10605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:24.015869 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10605
I20260926 20:46:24.016095 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:24.016680 5432 external_mini_cluster.cc:1428] Setting key 83b182ebb598b4e3049cb753b5877f83
I20260926 20:46:24.018738 10671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:24.032980 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:24.018845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.5.78.62:35731)
0926 20:46:24.019934 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:24.019948 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:24.020002 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:46:24.021977 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:24.022004 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:24.022700 (+ 696us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:24.023188 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:24.023776 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.025803 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.027415 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.028420 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.028609 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:24.029984 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:24.029994 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:24.030016 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:24.030846 (+ 830us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:24.030941 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:24.030968 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:24.031111 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:24.031343 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 20:46:24.031676 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":337,"threads_started":1}
I20260926 20:46:24.033384 10671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8: Bootstrap starting.
I20260926 20:46:24.036883 10671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:24.037107 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:46:24.037994 10671 log.cc:824] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:24.041121 10671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8: No bootstrap required, opened a new log
I20260926 20:46:24.049122 10671 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35731 } }
I20260926 20:46:24.049530 10671 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:24.049630 10671 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac8c0a5a76004d3c8ce7140ab4ab84b8, State: Initialized, Role: FOLLOWER
I20260926 20:46:24.050313 10671 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35731 } }
I20260926 20:46:24.050606 10671 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:24.050735 10671 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:24.050913 10671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:24.053337 10671 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35731 } }
I20260926 20:46:24.053978 10671 leader_election.cc:302] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac8c0a5a76004d3c8ce7140ab4ab84b8; no voters:
I20260926 20:46:24.054679 10671 leader_election.cc:288] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:24.054855 10676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:24.055614 10676 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [term 1 LEADER]: Becoming Leader. State: Replica: ac8c0a5a76004d3c8ce7140ab4ab84b8, State: Running, Role: LEADER
I20260926 20:46:24.056167 10676 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35731 } }
I20260926 20:46:24.056859 10671 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:24.059412 10677 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35731 } } }
I20260926 20:46:24.059870 10677 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:24.060365 10678 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac8c0a5a76004d3c8ce7140ab4ab84b8. Latest consensus state: current_term: 1 leader_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8c0a5a76004d3c8ce7140ab4ab84b8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35731 } } }
I20260926 20:46:24.060865 10678 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:24.062088 10682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:24.080358 10682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:24.092037 10682 catalog_manager.cc:1347] Generated new cluster ID: 32e136d8a2314b4a8f3e0b32c634e80f
I20260926 20:46:24.092180 10682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:24.108755 10682 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:24.110270 10682 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:24.120698 10682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac8c0a5a76004d3c8ce7140ab4ab84b8: Generated new TSK 0
I20260926 20:46:24.121595 10682 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 20:46:24.166574 10701 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 20:46:24.166764 10701 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 20:46:24.167881 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:24.142118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.5.78.62:35731)
0926 20:46:24.142423 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:24.142434 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:24.143047 (+ 613us) server_negotiation.cc:407] Connection header received
0926 20:46:24.143467 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:24.143471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:24.143549 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:24.143741 (+ 192us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:46:24.144317 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.145086 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.146592 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.147078 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.147222 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:24.147815 (+ 593us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:46:24.167160 (+ 19345us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 20:46:24.167553 (+ 393us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33714: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 20:46:24.168002 10674 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33714: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 20:46:24.169376 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:24.142105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:35731 (local address 127.0.0.1:33714)
0926 20:46:24.142906 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:24.142931 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:24.143147 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:24.143802 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:24.143807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:24.143828 (+ 21us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:46:24.144138 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:24.144147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.145248 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.145253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:24.146363 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:24.146375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.147215 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.147286 (+ 71us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:24.147437 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:24.147508 (+ 71us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:46:24.167491 (+ 19983us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 20:46:24.167813 (+ 322us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.78.62:35731: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":440,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.78.62:35731: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 20:46:24.171558 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10605
I20260926 20:46:24.172749 10666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:24.266628 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10605
2026-09-26T20:46:24Z chronyd exiting
I20260926 20:46:24.316932 5432 test_util.cc:175] -----------------------------------------------
I20260926 20:46:24.317129 5432 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455541382686-5432-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (768 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 20:46:24.321048 5432 mini_oidc.cc:152] Starting JWKS server
I20260926 20:46:24.323086 5432 webserver.cc:460] Webserver started at https://127.0.0.1:41503/ using document root <none> and password file <none>
I20260926 20:46:24.323391 5432 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 20:46:24.324095 5432 webserver.cc:460] Webserver started at http://127.0.0.1:37763/ using document root <none> and password file <none>
2026-09-26T20:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:24Z Disabled control of system clock
I20260926 20:46:24.351955 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46471
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:43783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:46471
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41503/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/testchainca.cert with env {}
W20260926 20:46:24.692416 10708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:24.692826 10708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:24.693014 10708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:24.700673 10708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:24.700872 10708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:24.700982 10708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:24.701107 10708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:24.701202 10708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:46:24.701303 10708 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:46:24.701390 10708 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41503/jwks.json
I20260926 20:46:24.712106 10708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:43783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:46471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41503/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:24.713550 10708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:24.715062 10708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:24.727875 10716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:24.727993 10714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:24.727955 10713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:24.729120 10708 server_base.cc:1034] running on GCE node
I20260926 20:46:24.729964 10708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:24.731513 10708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:24.732719 10708 hybrid_clock.cc:648] HybridClock initialized: now 1790455584732650 us; error 61 us; skew 500 ppm
I20260926 20:46:24.733094 10708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:24.741020 10708 webserver.cc:460] Webserver started at http://127.5.78.62:38201/ using document root <none> and password file <none>
I20260926 20:46:24.741802 10708 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:24.741935 10708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:24.742291 10708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:24.746138 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "9c619010c7d148479a20c322ca57d387"
format_stamp: "Formatted at 2026-09-26 20:46:24 on dist-test-slave-z7j8"
server_key: "cbb01152e5314151523a27f8bc0ea5a6"
server_key_iv: "cec723d4d45425f52c8ab942d8719b20"
server_key_version: "encryptionkey@0"
I20260926 20:46:24.747362 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "9c619010c7d148479a20c322ca57d387"
format_stamp: "Formatted at 2026-09-26 20:46:24 on dist-test-slave-z7j8"
server_key: "cbb01152e5314151523a27f8bc0ea5a6"
server_key_iv: "cec723d4d45425f52c8ab942d8719b20"
server_key_version: "encryptionkey@0"
I20260926 20:46:24.755177 10708 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 20:46:24.760066 10722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:24.761649 10708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260926 20:46:24.761883 10708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "9c619010c7d148479a20c322ca57d387"
format_stamp: "Formatted at 2026-09-26 20:46:24 on dist-test-slave-z7j8"
server_key: "cbb01152e5314151523a27f8bc0ea5a6"
server_key_iv: "cec723d4d45425f52c8ab942d8719b20"
server_key_version: "encryptionkey@0"
I20260926 20:46:24.762128 10708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:24.783157 10708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:24.784022 10708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:24.784312 10708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:24.803296 10708 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:46471
I20260926 20:46:24.803323 10773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:46471 every 8 connection(s)
I20260926 20:46:24.804657 10708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:24.809026 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:24.812552 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10708
I20260926 20:46:24.812733 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:24.813494 5432 external_mini_cluster.cc:1428] Setting key e19a3b78cf1b6b7b78100dd296248f8c
I20260926 20:46:24.823699 10774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387: Bootstrap starting.
I20260926 20:46:24.827514 10774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:24.828787 10774 log.cc:824] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:24.832068 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:24.815492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.5.78.62:46471)
0926 20:46:24.816778 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:24.816798 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:24.819046 (+ 2248us) server_negotiation.cc:407] Connection header received
0926 20:46:24.821891 (+ 2845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:24.821920 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:24.822536 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:24.822967 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:24.823525 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.825524 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.826921 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.828067 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.828254 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:24.829483 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:24.829500 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:24.829526 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:24.830317 (+ 791us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:24.830472 (+ 155us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:24.830503 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:24.830595 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:24.830816 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 20:46:24.831145 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":393,"threads_started":1}
I20260926 20:46:24.832077 10774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387: No bootstrap required, opened a new log
I20260926 20:46:24.835909 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:46:24.840916 10774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c619010c7d148479a20c322ca57d387" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46471 } }
I20260926 20:46:24.841259 10774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:24.841337 10774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c619010c7d148479a20c322ca57d387, State: Initialized, Role: FOLLOWER
I20260926 20:46:24.841827 10774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c619010c7d148479a20c322ca57d387" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46471 } }
I20260926 20:46:24.842013 10774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:24.842108 10774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:24.842226 10774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:24.844377 10774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c619010c7d148479a20c322ca57d387" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46471 } }
I20260926 20:46:24.844836 10774 leader_election.cc:302] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c619010c7d148479a20c322ca57d387; no voters:
I20260926 20:46:24.845531 10774 leader_election.cc:288] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:24.845732 10779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:24.846835 10779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [term 1 LEADER]: Becoming Leader. State: Replica: 9c619010c7d148479a20c322ca57d387, State: Running, Role: LEADER
I20260926 20:46:24.847354 10779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c619010c7d148479a20c322ca57d387" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46471 } }
I20260926 20:46:24.847772 10774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:24.850196 10781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c619010c7d148479a20c322ca57d387. Latest consensus state: current_term: 1 leader_uuid: "9c619010c7d148479a20c322ca57d387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c619010c7d148479a20c322ca57d387" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46471 } } }
I20260926 20:46:24.850682 10781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:24.851202 10780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c619010c7d148479a20c322ca57d387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c619010c7d148479a20c322ca57d387" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46471 } } }
I20260926 20:46:24.851500 10780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:24.853993 10788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:24.860500 10788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:24.870280 10788 catalog_manager.cc:1347] Generated new cluster ID: 16c209d1b5504b8a926061602612f024
I20260926 20:46:24.870440 10788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:24.904764 10788 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:24.906100 10788 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:24.911525 10788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c619010c7d148479a20c322ca57d387: Generated new TSK 0
I20260926 20:46:24.912352 10788 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 20:46:24.949088 10804 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 20:46:24.949225 10804 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 20:46:24.950397 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:24.929229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.5.78.62:46471)
0926 20:46:24.929451 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:24.929458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:24.929818 (+ 360us) server_negotiation.cc:407] Connection header received
0926 20:46:24.930074 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:24.930081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:24.930185 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:24.930387 (+ 202us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:46:24.930997 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.932203 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.933706 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.934515 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.934668 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:24.935238 (+ 570us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:46:24.949881 (+ 14643us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 20:46:24.950180 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":83}
W20260926 20:46:24.950603 10777 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 20:46:24.950935 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:24.929022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46471 (local address 127.0.0.1:60910)
0926 20:46:24.929703 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:24.929719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:24.929853 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:24.930522 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:24.930527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:24.930539 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:46:24.930805 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:24.930813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.932431 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.932436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:24.933535 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:24.933547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:24.934667 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:24.934715 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:24.934864 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:24.934905 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:46:24.950148 (+ 15243us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 20:46:24.950490 (+ 342us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.78.62:46471: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":388,"threads_started":1}
I20260926 20:46:24.952822 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10708
I20260926 20:46:24.954082 10769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:25.047561 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10708
2026-09-26T20:46:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (783 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 20:46:25.104625 5432 mini_oidc.cc:152] Starting JWKS server
I20260926 20:46:25.106559 5432 webserver.cc:460] Webserver started at https://127.0.0.1:45705/ using document root <none> and password file <none>
I20260926 20:46:25.106822 5432 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 20:46:25.107466 5432 webserver.cc:460] Webserver started at http://127.0.0.1:43637/ using document root <none> and password file <none>
2026-09-26T20:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:25Z Disabled control of system clock
I20260926 20:46:25.135210 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:35193
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:35193
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 20:46:25.457526 10811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:25.457816 10811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:25.457903 10811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:25.465181 10811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:25.465294 10811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:25.465333 10811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:25.465381 10811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:25.465420 10811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:46:25.465489 10811 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:46:25.465523 10811 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45705/jwks.json
I20260926 20:46:25.474732 10811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:35193
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:35193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:25.475980 10811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:25.477703 10811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:25.489917 10816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:25.489892 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:25.490321 10819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:25.491076 10811 server_base.cc:1034] running on GCE node
I20260926 20:46:25.491942 10811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:25.493438 10811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:25.494683 10811 hybrid_clock.cc:648] HybridClock initialized: now 1790455585494624 us; error 51 us; skew 500 ppm
I20260926 20:46:25.495055 10811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:25.497326 10811 webserver.cc:460] Webserver started at http://127.5.78.62:33009/ using document root <none> and password file <none>
I20260926 20:46:25.498055 10811 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:25.498193 10811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:25.498584 10811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:25.501559 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "9288a92b4d754dc1ba50d0ad55caa82d"
format_stamp: "Formatted at 2026-09-26 20:46:25 on dist-test-slave-z7j8"
server_key: "d795eb05d8f2d5cc71b6abb6e0f9670d"
server_key_iv: "564a655178011d026aa5e7ca01438d95"
server_key_version: "encryptionkey@0"
I20260926 20:46:25.502393 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "9288a92b4d754dc1ba50d0ad55caa82d"
format_stamp: "Formatted at 2026-09-26 20:46:25 on dist-test-slave-z7j8"
server_key: "d795eb05d8f2d5cc71b6abb6e0f9670d"
server_key_iv: "564a655178011d026aa5e7ca01438d95"
server_key_version: "encryptionkey@0"
I20260926 20:46:25.507722 10811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:46:25.511437 10825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:25.512849 10811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:46:25.513043 10811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "9288a92b4d754dc1ba50d0ad55caa82d"
format_stamp: "Formatted at 2026-09-26 20:46:25 on dist-test-slave-z7j8"
server_key: "d795eb05d8f2d5cc71b6abb6e0f9670d"
server_key_iv: "564a655178011d026aa5e7ca01438d95"
server_key_version: "encryptionkey@0"
I20260926 20:46:25.513267 10811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:25.539517 10811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:25.540407 10811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:25.540714 10811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:25.559571 10811 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:35193
I20260926 20:46:25.559587 10876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:35193 every 8 connection(s)
I20260926 20:46:25.561015 10811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:25.564617 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10811
I20260926 20:46:25.564839 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:25.565158 10877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:25.565564 5432 external_mini_cluster.cc:1428] Setting key fdbfc12ff2d8ffe65b9c819ccad34d27
I20260926 20:46:25.581784 10877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d: Bootstrap starting.
I20260926 20:46:25.582123 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:25.567551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.5.78.62:35193)
0926 20:46:25.568690 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:25.568706 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:25.568762 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:46:25.570618 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:25.570655 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:25.571354 (+ 699us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:25.571846 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:25.572501 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:25.574764 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:25.576243 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:25.577378 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:25.577612 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:25.578957 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:25.578973 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:25.578995 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:25.579812 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:25.579947 (+ 135us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:25.579981 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:25.580104 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:25.580362 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 20:46:25.580743 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":346,"threads_started":1}
I20260926 20:46:25.585506 10877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:25.586535 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:46:25.586724 10877 log.cc:824] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:25.590104 10877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d: No bootstrap required, opened a new log
I20260926 20:46:25.597929 10877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35193 } }
I20260926 20:46:25.598516 10877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:25.598672 10877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9288a92b4d754dc1ba50d0ad55caa82d, State: Initialized, Role: FOLLOWER
I20260926 20:46:25.599334 10877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35193 } }
I20260926 20:46:25.599602 10877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:25.599764 10877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:25.599977 10877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:25.602294 10877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35193 } }
I20260926 20:46:25.602931 10877 leader_election.cc:302] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9288a92b4d754dc1ba50d0ad55caa82d; no voters:
I20260926 20:46:25.603505 10877 leader_election.cc:288] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:25.603708 10882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:25.604558 10882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [term 1 LEADER]: Becoming Leader. State: Replica: 9288a92b4d754dc1ba50d0ad55caa82d, State: Running, Role: LEADER
I20260926 20:46:25.605297 10882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35193 } }
I20260926 20:46:25.606315 10877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:25.608979 10883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35193 } } }
I20260926 20:46:25.609537 10883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:25.609737 10884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9288a92b4d754dc1ba50d0ad55caa82d. Latest consensus state: current_term: 1 leader_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9288a92b4d754dc1ba50d0ad55caa82d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35193 } } }
I20260926 20:46:25.610075 10884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:25.611840 10891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:25.616962 10891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:25.625737 10891 catalog_manager.cc:1347] Generated new cluster ID: 0a736ad751d64dfbacd3e9eef603cdb7
I20260926 20:46:25.625867 10891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:25.639098 10891 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:25.640475 10891 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:25.646492 10891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9288a92b4d754dc1ba50d0ad55caa82d: Generated new TSK 0
I20260926 20:46:25.647467 10891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:46:25.667693 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:25.659907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.5.78.62:35193)
0926 20:46:25.660084 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:25.660108 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:25.660605 (+ 497us) server_negotiation.cc:407] Connection header received
0926 20:46:25.660887 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:25.660891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:25.660965 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:25.661141 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:46:25.661614 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:25.662350 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:25.663829 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:25.664288 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:25.664424 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:25.664969 (+ 545us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:46:25.667012 (+ 2043us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 20:46:25.667357 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55890: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 20:46:25.667863 10906 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:25.659838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:35193 (local address 127.0.0.1:55890)
0926 20:46:25.660482 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:25.660499 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:25.660685 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:25.661153 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:25.661158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:25.661170 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:46:25.661457 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:25.661466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:25.662547 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:25.662554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:25.663671 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:25.663683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:25.664411 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:25.664491 (+ 80us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:25.664650 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:25.664686 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:46:25.667225 (+ 2539us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 20:46:25.667463 (+ 238us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.78.62:35193: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":372,"threads_started":1}
W20260926 20:46:25.668079 10880 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55890: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260926 20:46:25.668965 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10811
I20260926 20:46:25.670217 10872 generic_service.cc:196] Checking for leaks (request via RPC)
E20260926 20:46:25.679965 10907 webserver.cc:533] Webserver: sslize failed:
E20260926 20:46:25.680094 10907 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 20:46:25.759785 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10811
2026-09-26T20:46:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (713 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T20:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:25Z Disabled control of system clock
I20260926 20:46:25.847297 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:41705
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:41705
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/testchainca.cert with env {}
W20260926 20:46:26.178901 10912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:26.179214 10912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:26.179299 10912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:26.187182 10912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:26.187301 10912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:26.187338 10912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:26.187394 10912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:26.187438 10912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:46:26.187498 10912 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:46:26.187541 10912 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 20:46:26.198302 10912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:41705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:26.200102 10912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:26.201642 10912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:26.214205 10917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:26.214133 10920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:26.214162 10918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:26.215061 10912 server_base.cc:1034] running on GCE node
I20260926 20:46:26.215942 10912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:26.217413 10912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:26.218729 10912 hybrid_clock.cc:648] HybridClock initialized: now 1790455586218676 us; error 37 us; skew 500 ppm
I20260926 20:46:26.219129 10912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:46:26.221547 10912 webserver.cc:460] Webserver started at http://127.5.78.62:34135/ using document root <none> and password file <none>
I20260926 20:46:26.222287 10912 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:26.222419 10912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:26.222796 10912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:26.225728 10912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "b28938504d0d437eabfbcc5cb1a00426"
format_stamp: "Formatted at 2026-09-26 20:46:26 on dist-test-slave-z7j8"
server_key: "6e1ede03e2dc0fe731a5abdb4b985626"
server_key_iv: "7de0cc43ebc377d7240a3662a2b12e18"
server_key_version: "encryptionkey@0"
I20260926 20:46:26.226631 10912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "b28938504d0d437eabfbcc5cb1a00426"
format_stamp: "Formatted at 2026-09-26 20:46:26 on dist-test-slave-z7j8"
server_key: "6e1ede03e2dc0fe731a5abdb4b985626"
server_key_iv: "7de0cc43ebc377d7240a3662a2b12e18"
server_key_version: "encryptionkey@0"
I20260926 20:46:26.232255 10912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:46:26.236160 10926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:26.237879 10912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:26.238121 10912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "b28938504d0d437eabfbcc5cb1a00426"
format_stamp: "Formatted at 2026-09-26 20:46:26 on dist-test-slave-z7j8"
server_key: "6e1ede03e2dc0fe731a5abdb4b985626"
server_key_iv: "7de0cc43ebc377d7240a3662a2b12e18"
server_key_version: "encryptionkey@0"
I20260926 20:46:26.238380 10912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:26.256758 10912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:26.257687 10912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:26.257997 10912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:26.276974 10912 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:41705
I20260926 20:46:26.276981 10977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:41705 every 8 connection(s)
I20260926 20:46:26.278431 10912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:26.282348 10978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:26.288591 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 10912
I20260926 20:46:26.288822 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:26.289449 5432 external_mini_cluster.cc:1428] Setting key 4434f429c8f625cd1b8f81f161b27c0c
I20260926 20:46:26.295027 10978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426: Bootstrap starting.
I20260926 20:46:26.298172 10978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:26.299259 10978 log.cc:824] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:26.302273 10978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426: No bootstrap required, opened a new log
I20260926 20:46:26.303468 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:26.291401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.5.78.62:41705)
0926 20:46:26.292686 (+ 1285us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:26.292703 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:26.292760 (+ 57us) server_negotiation.cc:407] Connection header received
0926 20:46:26.294674 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:26.294700 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:26.295234 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:26.295661 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:26.296226 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:26.297550 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:26.299008 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:26.299899 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:26.300106 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:26.301201 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:26.301208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:26.301222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:46:26.301749 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:26.301828 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:46:26.301847 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:26.301978 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:26.302192 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 20:46:26.302532 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":407,"threads_started":1}
I20260926 20:46:26.307636 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:46:26.311162 10978 raft_consensus.cc:348] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28938504d0d437eabfbcc5cb1a00426" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 41705 } }
I20260926 20:46:26.311532 10978 raft_consensus.cc:374] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:26.311632 10978 raft_consensus.cc:729] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b28938504d0d437eabfbcc5cb1a00426, State: Initialized, Role: FOLLOWER
I20260926 20:46:26.312281 10978 consensus_queue.cc:260] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28938504d0d437eabfbcc5cb1a00426" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 41705 } }
I20260926 20:46:26.312579 10978 raft_consensus.cc:388] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:26.312749 10978 raft_consensus.cc:482] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:26.312934 10978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:26.315227 10978 raft_consensus.cc:504] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28938504d0d437eabfbcc5cb1a00426" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 41705 } }
I20260926 20:46:26.315724 10978 leader_election.cc:302] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b28938504d0d437eabfbcc5cb1a00426; no voters:
I20260926 20:46:26.316303 10978 leader_election.cc:288] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:26.316504 10983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:26.317252 10983 raft_consensus.cc:686] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [term 1 LEADER]: Becoming Leader. State: Replica: b28938504d0d437eabfbcc5cb1a00426, State: Running, Role: LEADER
I20260926 20:46:26.317792 10983 consensus_queue.cc:237] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28938504d0d437eabfbcc5cb1a00426" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 41705 } }
I20260926 20:46:26.318451 10978 sys_catalog.cc:564] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:26.320868 10985 sys_catalog.cc:455] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b28938504d0d437eabfbcc5cb1a00426. Latest consensus state: current_term: 1 leader_uuid: "b28938504d0d437eabfbcc5cb1a00426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28938504d0d437eabfbcc5cb1a00426" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 41705 } } }
I20260926 20:46:26.321403 10985 sys_catalog.cc:458] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:26.321004 10984 sys_catalog.cc:455] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b28938504d0d437eabfbcc5cb1a00426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28938504d0d437eabfbcc5cb1a00426" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 41705 } } }
I20260926 20:46:26.321705 10984 sys_catalog.cc:458] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:26.322731 10990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:26.331439 10990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:26.341048 10990 catalog_manager.cc:1347] Generated new cluster ID: aabe56f28c1040f4b0eee0cc13fd0d98
I20260926 20:46:26.341199 10990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:26.367360 10990 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:26.368688 10990 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:26.377518 10990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b28938504d0d437eabfbcc5cb1a00426: Generated new TSK 0
I20260926 20:46:26.378710 10990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:46:26.453253 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:26.402312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.5.78.62:41705)
0926 20:46:26.402536 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:26.402543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:26.402942 (+ 399us) server_negotiation.cc:407] Connection header received
0926 20:46:26.403177 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:26.403183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:26.403272 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:26.403470 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:46:26.403893 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:26.405020 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:26.406456 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:26.407152 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:26.407278 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:26.408536 (+ 1258us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:46:26.452394 (+ 43858us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 20:46:26.452919 (+ 525us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55814: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 20:46:26.453398 10981 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55814: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 20:46:26.453512 11007 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:26.402164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:41705 (local address 127.0.0.1:55814)
0926 20:46:26.402843 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:26.402858 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:26.402979 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:26.403475 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:26.403478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:26.403487 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:46:26.403748 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:26.403755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:26.405178 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:26.405182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:26.406281 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:26.406292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:26.408030 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:26.408076 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:26.408216 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:26.408256 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:46:26.452786 (+ 44530us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 20:46:26.453065 (+ 279us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.78.62:41705: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":405,"threads_started":1}
I20260926 20:46:26.455000 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 10912
I20260926 20:46:26.456204 10973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:26.553838 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 10912
2026-09-26T20:46:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (770 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (419 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (426 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:27 dist-test-slave-z7j8 krb5kdc[11022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:27 dist-test-slave-z7j8 krb5kdc[11022](info): set up 2 sockets
Sep 26 20:46:27 dist-test-slave-z7j8 krb5kdc[11022](info): commencing operation
krb5kdc: starting...
W20260926 20:46:28.487136 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:28 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:28.698902 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:39165
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:39165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:29.032508 11035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:29.032833 11035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:29.033025 11035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:29.040661 11035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:29.040794 11035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:29.040843 11035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:29.040949 11035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:29.041015 11035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:29.051362 11035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:39165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:39165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:29.052778 11035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:29.054157 11035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:29.067525 11043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:29.067669 11040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:29.067679 11041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:29.068831 11035 server_base.cc:1034] running on GCE node
I20260926 20:46:29.070116 11035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:29.072279 11035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:29.073562 11035 hybrid_clock.cc:648] HybridClock initialized: now 1790455589073469 us; error 69 us; skew 500 ppm
I20260926 20:46:29.074086 11035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:29 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455589, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:29.077761 11035 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:29.079447 11035 webserver.cc:460] Webserver started at http://127.5.78.62:46453/ using document root <none> and password file <none>
I20260926 20:46:29.080431 11035 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:29.080623 11035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:29.080952 11035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:29.084447 11035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "49a9e7fde65a46e28f0cbc5c935721bb"
format_stamp: "Formatted at 2026-09-26 20:46:29 on dist-test-slave-z7j8"
server_key: "2fd645f8b66f50837af19697dd010272"
server_key_iv: "245315022598ff3d162235c604881fd8"
server_key_version: "encryptionkey@0"
I20260926 20:46:29.085300 11035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "49a9e7fde65a46e28f0cbc5c935721bb"
format_stamp: "Formatted at 2026-09-26 20:46:29 on dist-test-slave-z7j8"
server_key: "2fd645f8b66f50837af19697dd010272"
server_key_iv: "245315022598ff3d162235c604881fd8"
server_key_version: "encryptionkey@0"
I20260926 20:46:29.091490 11035 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260926 20:46:29.096287 11050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:29.097774 11035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:29.097965 11035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "49a9e7fde65a46e28f0cbc5c935721bb"
format_stamp: "Formatted at 2026-09-26 20:46:29 on dist-test-slave-z7j8"
server_key: "2fd645f8b66f50837af19697dd010272"
server_key_iv: "245315022598ff3d162235c604881fd8"
server_key_version: "encryptionkey@0"
I20260926 20:46:29.098191 11035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:29.130611 11035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:29.135213 11035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:29.135526 11035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:29.154848 11035 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:39165
I20260926 20:46:29.154872 11101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:39165 every 8 connection(s)
I20260926 20:46:29.156244 11035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:29.160049 11102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:29.160377 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11035
I20260926 20:46:29.160576 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:29.161206 5432 external_mini_cluster.cc:1428] Setting key 05fc6fd29c457aa950dbbcbdf72b2858
I20260926 20:46:29.172828 11102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb: Bootstrap starting.
Sep 26 20:46:29 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:29.176405 11102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:29.177516 11102 log.cc:824] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:29.180574 11102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb: No bootstrap required, opened a new log
I20260926 20:46:29.184425 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:29.163292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.5.78.62:39165)
0926 20:46:29.164153 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:29.164167 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:29.164212 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:46:29.166319 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:29.166347 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:29.166931 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:29.167397 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:29.168876 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:29.170227 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:29.171530 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:29.172244 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:29.172447 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:29.176899 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:29.176908 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:29.176924 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:29.176964 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:29.180146 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:29.180981 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:29.180988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:29.180994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:29.181170 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:29.181575 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:29.181586 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:29.181588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:29.182130 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:29.182494 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:29.182929 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 20:46:29.183371 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":394,"threads_started":1}
I20260926 20:46:29.188839 11102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39165 } }
I20260926 20:46:29.189224 11102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:29.189369 11102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a9e7fde65a46e28f0cbc5c935721bb, State: Initialized, Role: FOLLOWER
I20260926 20:46:29.189966 11102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39165 } }
I20260926 20:46:29.190246 11102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:29.190392 11102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:29.190629 11102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:29.192914 11102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39165 } }
I20260926 20:46:29.193526 11102 leader_election.cc:302] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49a9e7fde65a46e28f0cbc5c935721bb; no voters:
I20260926 20:46:29.194123 11102 leader_election.cc:288] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:29.194305 11107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:29.195835 11107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [term 1 LEADER]: Becoming Leader. State: Replica: 49a9e7fde65a46e28f0cbc5c935721bb, State: Running, Role: LEADER
I20260926 20:46:29.196363 11102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:29.196329 11107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39165 } }
I20260926 20:46:29.199513 11108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39165 } } }
I20260926 20:46:29.199484 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49a9e7fde65a46e28f0cbc5c935721bb. Latest consensus state: current_term: 1 leader_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a9e7fde65a46e28f0cbc5c935721bb" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39165 } } }
I20260926 20:46:29.199966 11108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:29.200013 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:29.201390 11115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:29.206130 11115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:29.214771 11115 catalog_manager.cc:1347] Generated new cluster ID: 5be44ba73d6345a89bba4b6d20baee9d
I20260926 20:46:29.214908 11115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:29.225910 11115 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:29.227948 11115 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:29.237035 11115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49a9e7fde65a46e28f0cbc5c935721bb: Generated new TSK 0
I20260926 20:46:29.238111 11115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:46:29.297948 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39165
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:29.632417 11128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:29.632706 11128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:29.632895 11128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:29.640376 11128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:29.640516 11128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:29.640712 11128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:29.650801 11128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:29.652125 11128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:29.653502 11128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:29.666616 11133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:29.666577 11136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:29.666661 11134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:29.667439 11128 server_base.cc:1034] running on GCE node
I20260926 20:46:29.668136 11128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:29.669142 11128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:29.670373 11128 hybrid_clock.cc:648] HybridClock initialized: now 1790455589670313 us; error 56 us; skew 500 ppm
I20260926 20:46:29.670739 11128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:29 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455589, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:29.673841 11128 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:46:29.675386 11128 webserver.cc:460] Webserver started at http://127.5.78.1:39053/ using document root <none> and password file <none>
I20260926 20:46:29.676041 11128 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:29.676141 11128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:29.676388 11128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:29.679539 11128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "e1cd2fad4179412eb5c7aba5d55bc882"
format_stamp: "Formatted at 2026-09-26 20:46:29 on dist-test-slave-z7j8"
server_key: "3bf697288a04a42690ec570a8e094202"
server_key_iv: "34da587100cacb265eb4f2d2eb4362f2"
server_key_version: "encryptionkey@0"
I20260926 20:46:29.680323 11128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "e1cd2fad4179412eb5c7aba5d55bc882"
format_stamp: "Formatted at 2026-09-26 20:46:29 on dist-test-slave-z7j8"
server_key: "3bf697288a04a42690ec570a8e094202"
server_key_iv: "34da587100cacb265eb4f2d2eb4362f2"
server_key_version: "encryptionkey@0"
I20260926 20:46:29.685513 11128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:29.688942 11143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:29.690255 11128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:46:29.690383 11128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "e1cd2fad4179412eb5c7aba5d55bc882"
format_stamp: "Formatted at 2026-09-26 20:46:29 on dist-test-slave-z7j8"
server_key: "3bf697288a04a42690ec570a8e094202"
server_key_iv: "34da587100cacb265eb4f2d2eb4362f2"
server_key_version: "encryptionkey@0"
I20260926 20:46:29.690624 11128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:29.704419 11128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:29.708467 11128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:29.708706 11128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:29.709578 11128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:29.710857 11128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:29.710930 11128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:29.711009 11128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:29.711055 11128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:29.744454 11128 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:39681
I20260926 20:46:29.744499 11255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:39681 every 8 connection(s)
I20260926 20:46:29.745826 11128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:29.749409 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11128
I20260926 20:46:29.749676 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:29.750488 5432 external_mini_cluster.cc:1428] Setting key 11dcbd02a02e8e0cbac67d20a4236828
Sep 26 20:46:29 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455589, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:46:29.770644 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:29.748438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:46681 (local address 127.5.78.62:39165)
0926 20:46:29.748903 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:29.748910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:29.750064 (+ 1154us) server_negotiation.cc:407] Connection header received
0926 20:46:29.752240 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:29.752248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:29.752365 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:29.752452 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:29.753985 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:29.755162 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:29.756695 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:29.757391 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:29.757553 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:29.764659 (+ 7106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:29.764665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:29.764668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:29.764704 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:29.767478 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:29.768280 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:29.768285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:29.768288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:29.768404 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:29.768857 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:29.768864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:29.768868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:29.769142 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:29.769299 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:29.770118 (+ 819us) server_negotiation.cc:299] Negotiation successful
0926 20:46:29.770314 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":218,"threads_started":1}
I20260926 20:46:29.771718 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:29.749028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39165 (local address 127.5.78.1:46681)
0926 20:46:29.749859 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:29.749911 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:29.751920 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:29.752711 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:29.752720 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:29.753235 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:29.753808 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:29.753824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:29.755307 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:29.755310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:29.756551 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:29.756563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:29.757610 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:29.757615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:29.757813 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:29.761891 (+ 4078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:29.761931 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:29.764373 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:29.767649 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:29.767659 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:29.767678 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:29.768126 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:29.768527 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:29.768532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:29.768536 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:29.768740 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:29.769275 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:29.769283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:29.769655 (+ 372us) client_negotiation.cc:769] Sending connection context
0926 20:46:29.769946 (+ 291us) client_negotiation.cc:240] Negotiation successful
0926 20:46:29.770368 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":355,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:46:29.773646 11256 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39165
I20260926 20:46:29.773979 11256 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:29.774827 11256 heartbeater.cc:511] Master 127.5.78.62:39165 requested a full tablet report, sending...
I20260926 20:46:29.776971 11067 ts_manager.cc:194] Registered new tserver with Master: e1cd2fad4179412eb5c7aba5d55bc882 (127.5.78.1:39681)
I20260926 20:46:29.778491 11067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:46681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:46:29.798493 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39165
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:30.131644 11262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:30.131969 11262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:30.132067 11262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:30.139577 11262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:30.139700 11262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:30.139820 11262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:30.149906 11262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:30.151279 11262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:30.152514 11262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:30.166867 11268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:30.167181 11270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:30.166873 11267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:30.168020 11262 server_base.cc:1034] running on GCE node
I20260926 20:46:30.168744 11262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:30.169731 11262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:30.170966 11262 hybrid_clock.cc:648] HybridClock initialized: now 1790455590170913 us; error 47 us; skew 500 ppm
I20260926 20:46:30.171306 11262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.174536 11262 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:46:30.175992 11262 webserver.cc:460] Webserver started at http://127.5.78.2:41421/ using document root <none> and password file <none>
I20260926 20:46:30.176681 11262 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:30.176820 11262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:30.177132 11262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:30.180085 11262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "4eca146ae32247a4b27b9a4af4f7e5bc"
format_stamp: "Formatted at 2026-09-26 20:46:30 on dist-test-slave-z7j8"
server_key: "4326059cd8a57e71d493decc79177b84"
server_key_iv: "f64db7f7bb360c5e87191396b34506dd"
server_key_version: "encryptionkey@0"
I20260926 20:46:30.180830 11262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "4eca146ae32247a4b27b9a4af4f7e5bc"
format_stamp: "Formatted at 2026-09-26 20:46:30 on dist-test-slave-z7j8"
server_key: "4326059cd8a57e71d493decc79177b84"
server_key_iv: "f64db7f7bb360c5e87191396b34506dd"
server_key_version: "encryptionkey@0"
I20260926 20:46:30.185889 11262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:46:30.189679 11277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:30.191358 11262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:30.191550 11262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "4eca146ae32247a4b27b9a4af4f7e5bc"
format_stamp: "Formatted at 2026-09-26 20:46:30 on dist-test-slave-z7j8"
server_key: "4326059cd8a57e71d493decc79177b84"
server_key_iv: "f64db7f7bb360c5e87191396b34506dd"
server_key_version: "encryptionkey@0"
I20260926 20:46:30.191773 11262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:30.207330 11262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:30.211695 11262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:30.211993 11262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:30.213001 11262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:30.214372 11262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:30.214524 11262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:30.214684 11262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:30.214771 11262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:30.251253 11262 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:34175
I20260926 20:46:30.251288 11389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:34175 every 8 connection(s)
I20260926 20:46:30.252609 11262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:30.260192 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11262
I20260926 20:46:30.260391 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:30.260810 5432 external_mini_cluster.cc:1428] Setting key 690c2fb6f28f545bfeb9f4e6533d51ae
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:30.273372 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.255316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:50933 (local address 127.5.78.62:39165)
0926 20:46:30.255527 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.255532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.256741 (+ 1209us) server_negotiation.cc:407] Connection header received
0926 20:46:30.258589 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.258595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.258693 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.258808 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.260405 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.261237 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.263318 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.263740 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.263878 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.267588 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:30.267593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:30.267597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:30.267628 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:30.270217 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.270982 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.270991 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.270994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.271127 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.271569 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.271574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.271577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.271832 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:30.271986 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:30.272810 (+ 824us) server_negotiation.cc:299] Negotiation successful
0926 20:46:30.273028 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 20:46:30.274469 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.255767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39165 (local address 127.5.78.2:50933)
0926 20:46:30.256569 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:30.256608 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:30.258255 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:30.259060 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:30.259068 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:30.259600 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:30.260218 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:30.260236 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.261405 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.261410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:30.263173 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:30.263186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.263969 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.263976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:30.264210 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.264924 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:30.264949 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:30.267358 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:30.270367 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:30.270377 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:30.270394 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:30.270855 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:30.271240 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:30.271247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:30.271250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:30.271446 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:30.271959 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:30.271967 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:30.272390 (+ 423us) client_negotiation.cc:769] Sending connection context
0926 20:46:30.272714 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 20:46:30.273251 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":350,"threads_started":1}
I20260926 20:46:30.275959 11390 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39165
I20260926 20:46:30.276289 11390 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:30.277016 11390 heartbeater.cc:511] Master 127.5.78.62:39165 requested a full tablet report, sending...
I20260926 20:46:30.278493 11067 ts_manager.cc:194] Registered new tserver with Master: 4eca146ae32247a4b27b9a4af4f7e5bc (127.5.78.2:34175)
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:46:30.279377 11067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:50933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:46:30.306069 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39165
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:30.627893 11395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:30.628247 11395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:30.628422 11395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:30.636144 11395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:30.636322 11395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:30.636513 11395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:30.646587 11395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:30.648031 11395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:30.649449 11395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:30.663515 11400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:30.663560 11401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:30.663483 11403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:30.664984 11395 server_base.cc:1034] running on GCE node
I20260926 20:46:30.665660 11395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:30.666750 11395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:30.667989 11395 hybrid_clock.cc:648] HybridClock initialized: now 1790455590667942 us; error 44 us; skew 500 ppm
I20260926 20:46:30.668318 11395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.674927 11395 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:46:30.676607 11395 webserver.cc:460] Webserver started at http://127.5.78.3:42483/ using document root <none> and password file <none>
I20260926 20:46:30.677351 11395 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:30.677503 11395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:30.677834 11395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:30.681129 11395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "d674eb05b9374885b84bf957125e4319"
format_stamp: "Formatted at 2026-09-26 20:46:30 on dist-test-slave-z7j8"
server_key: "455fe8c388d7f5b0af0fbcece49b8fd0"
server_key_iv: "1b089ac83a3383d630f12aa9e4850383"
server_key_version: "encryptionkey@0"
I20260926 20:46:30.682090 11395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "d674eb05b9374885b84bf957125e4319"
format_stamp: "Formatted at 2026-09-26 20:46:30 on dist-test-slave-z7j8"
server_key: "455fe8c388d7f5b0af0fbcece49b8fd0"
server_key_iv: "1b089ac83a3383d630f12aa9e4850383"
server_key_version: "encryptionkey@0"
I20260926 20:46:30.688027 11395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:46:30.692222 11410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:30.694098 11395 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 20:46:30.694317 11395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "d674eb05b9374885b84bf957125e4319"
format_stamp: "Formatted at 2026-09-26 20:46:30 on dist-test-slave-z7j8"
server_key: "455fe8c388d7f5b0af0fbcece49b8fd0"
server_key_iv: "1b089ac83a3383d630f12aa9e4850383"
server_key_version: "encryptionkey@0"
I20260926 20:46:30.694643 11395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:30.708997 11395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:30.713464 11395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:30.713717 11395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:30.714728 11395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:30.716184 11395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:30.716259 11395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:30.716379 11395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:30.716423 11395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:30.767213 11395 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:40909
I20260926 20:46:30.767333 11522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:40909 every 8 connection(s)
I20260926 20:46:30.768913 11395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:30.776638 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11395
I20260926 20:46:30.776841 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:30.777455 5432 external_mini_cluster.cc:1428] Setting key 6f75c2e9a2fddf9a852596c6ceb1a5fa
I20260926 20:46:30.781616 11256 heartbeater.cc:503] Master 127.5.78.62:39165 was elected leader, sending a full tablet report...
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:30.799013 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.772151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:37245 (local address 127.5.78.62:39165)
0926 20:46:30.773660 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.773671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.774466 (+ 795us) server_negotiation.cc:407] Connection header received
0926 20:46:30.780162 (+ 5696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.780167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.780250 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.780355 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.782951 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.784262 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.787233 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.787612 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.787714 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.791829 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:30.791835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:30.791838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:30.791870 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:30.795233 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.796164 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.796169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.796172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.796332 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.796961 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.796966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.796970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.797264 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:30.797429 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:30.798308 (+ 879us) server_negotiation.cc:299] Negotiation successful
0926 20:46:30.798591 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313}
I20260926 20:46:30.800035 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.772999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39165 (local address 127.5.78.3:37245)
0926 20:46:30.774083 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:30.774148 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:30.779804 (+ 5656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:30.780694 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:30.780710 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:30.781706 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:30.782753 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:30.782794 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.784437 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.784450 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:30.787061 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:30.787085 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.787857 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.787866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:30.788241 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.789161 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:30.789186 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:30.791564 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:30.795393 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:30.795403 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:30.795418 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:30.796007 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:30.796477 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:30.796481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:30.796483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:30.796823 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:30.797403 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:30.797411 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:30.797897 (+ 486us) client_negotiation.cc:769] Sending connection context
0926 20:46:30.798183 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 20:46:30.798840 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":529,"threads_started":1}
I20260926 20:46:30.801717 11523 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39165
I20260926 20:46:30.802070 11523 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:30.803012 11523 heartbeater.cc:511] Master 127.5.78.62:39165 requested a full tablet report, sending...
I20260926 20:46:30.804700 11067 ts_manager.cc:194] Registered new tserver with Master: d674eb05b9374885b84bf957125e4319 (127.5.78.3:40909)
I20260926 20:46:30.805876 11067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:37245
I20260926 20:46:30.806180 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/test-admin.keytab.
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.834476 5432 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 20:46:30.840071 11530 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:30.837406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49486 (local address 127.5.78.1:39681)
0926 20:46:30.838100 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.838107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.838133 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:46:30.838208 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.838215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.838494 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.838662 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.839596 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49486: BlockingRecv error: recv got EOF from 127.0.0.1:49486 (error 108)
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":320,"threads_started":1}
W20260926 20:46:30.840390 5432 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc)
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.842043 5432 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.843611 5432 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:30.859517 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.845326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.5.78.1:39681)
0926 20:46:30.845548 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.845555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.845808 (+ 253us) server_negotiation.cc:407] Connection header received
0926 20:46:30.846020 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.846025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.846098 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.846212 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.847270 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.848239 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.849482 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.850070 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.850231 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.853985 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:30.853997 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:30.854013 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:30.854045 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:30.856986 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.857631 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.857639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.857641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.857763 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.858075 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.858078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.858080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.858484 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:30.858694 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:30.858966 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 20:46:30.859260 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 20:46:30.865526 11530 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:30.863902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49498 (local address 127.5.78.1:39681)
0926 20:46:30.864076 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.864083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.864304 (+ 221us) server_negotiation.cc:407] Connection header received
0926 20:46:30.864432 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.864435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.864501 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.864582 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.865243 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49498: BlockingRecv error: recv got EOF from 127.0.0.1:49498 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 20:46:30.865731 5432 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc' of type FILE
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.867595 5432 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.869189 5432 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:30.884698 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.870670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.5.78.1:39681)
0926 20:46:30.870872 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.870877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.871327 (+ 450us) server_negotiation.cc:407] Connection header received
0926 20:46:30.871461 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.871465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.871523 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.871606 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.872626 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.873388 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.874781 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.875272 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.875393 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.879307 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:30.879313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:30.879317 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:30.879378 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:30.881968 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.882750 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.882754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.882757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.882890 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.883327 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.883331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.883333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.883601 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:30.883799 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:30.884174 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 20:46:30.884360 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 20:46:30.889520 11530 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:30.887670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.5.78.1:39681)
0926 20:46:30.887886 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.887892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.888129 (+ 237us) server_negotiation.cc:407] Connection header received
0926 20:46:30.888334 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.888343 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.888412 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.888516 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.889251 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49524: BlockingRecv error: recv got EOF from 127.0.0.1:49524 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260926 20:46:30.889835 5432 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455541382686-5432-0/krb5kdc/krb5cc' of type FILE
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.891464 5432 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:30.892804 5432 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 20:46:30 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:30.908509 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.894235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.5.78.1:39681)
0926 20:46:30.894539 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.894545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.894704 (+ 159us) server_negotiation.cc:407] Connection header received
0926 20:46:30.894848 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.894852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.894968 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.895083 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.896143 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.896851 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.898252 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.898779 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.898918 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.903102 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:30.903108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:30.903112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:30.903171 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:30.906061 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.906820 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.906825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.906827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.906981 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.907346 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.907350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.907352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.907607 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:30.907833 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:30.908073 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 20:46:30.908260 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260926 20:46:30.910086 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11128
I20260926 20:46:30.922976 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:30.910975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.5.78.1:39681)
0926 20:46:30.911172 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:30.911178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:30.911470 (+ 292us) server_negotiation.cc:407] Connection header received
0926 20:46:30.911681 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:30.911686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:30.911747 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:30.911849 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:30.912771 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.913446 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.914785 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:30.915213 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:30.915358 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:30.917656 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:30.917662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:30.917665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:30.917701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:30.920354 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.921044 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.921051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.921055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.921202 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:30.921559 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:30.921566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:30.921569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:30.921833 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:30.922093 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:30.922443 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 20:46:30.922658 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:46:30.923638 11251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:31.009495 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11128
I20260926 20:46:31.026598 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11262
Sep 26 20:46:31 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:46:31.042827 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:31.027500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.5.78.2:34175)
0926 20:46:31.028055 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:31.028065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:31.028091 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:46:31.028151 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:31.028157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:31.028379 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:31.028517 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:31.029476 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:31.030568 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:31.031738 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:31.032280 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:31.032375 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:31.036400 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:31.036414 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:31.036430 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:31.036467 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:31.039727 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:31.040627 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:31.040644 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:31.040646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:31.040780 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:31.041177 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:31.041186 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:31.041189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:31.041575 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:31.041918 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:31.042226 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 20:46:31.042515 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":290,"threads_started":1}
I20260926 20:46:31.044104 11385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:31.136513 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11262
I20260926 20:46:31.152211 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11395
Sep 26 20:46:31 dist-test-slave-z7j8 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:46:31.169687 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:31.153403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.5.78.3:40909)
0926 20:46:31.154104 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:31.154111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:31.154152 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:46:31.154216 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:31.154223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:31.154537 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:31.154700 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:31.155947 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:31.156971 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:31.158797 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:31.159366 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:31.159465 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:31.164032 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:31.164044 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:31.164058 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:31.164089 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:31.167005 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:31.167691 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:31.167701 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:31.167703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:31.167819 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:31.168187 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:31.168193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:31.168210 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:31.168534 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:31.168862 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:31.169149 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 20:46:31.169408 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":393,"threads_started":1}
I20260926 20:46:31.171092 11518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:31.253643 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11395
I20260926 20:46:31.269371 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11035
I20260926 20:46:31.270778 11097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:31.372807 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11035
2026-09-26T20:46:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3979 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:31 dist-test-slave-z7j8 krb5kdc[11550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:31 dist-test-slave-z7j8 krb5kdc[11550](info): set up 2 sockets
Sep 26 20:46:31 dist-test-slave-z7j8 krb5kdc[11550](info): commencing operation
krb5kdc: starting...
W20260926 20:46:32.495354 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:32 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab.
Entry for principal oryx/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab.
I20260926 20:46:32.711374 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38333
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab
--principal=oryx/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:38333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:33.034471 11563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:33.034860 11563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:33.035038 11563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:33.042661 11563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:33.042831 11563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:33.042944 11563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:33.043051 11563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:33.043136 11563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:33.053129 11563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:38333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab
--principal=oryx/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:33.054575 11563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:33.055962 11563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:33.070809 11568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:33.071038 11571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:33.071051 11569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:33.071759 11563 server_base.cc:1034] running on GCE node
I20260926 20:46:33.072728 11563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:33.074482 11563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:33.075755 11563 hybrid_clock.cc:648] HybridClock initialized: now 1790455593075697 us; error 51 us; skew 500 ppm
I20260926 20:46:33.076206 11563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:33 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455593, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:33.079877 11563 init.cc:377] Logged in from keytab as oryx/127.5.78.62@KRBTEST.COM (short username oryx)
I20260926 20:46:33.081658 11563 webserver.cc:460] Webserver started at http://127.5.78.62:44439/ using document root <none> and password file <none>
I20260926 20:46:33.082506 11563 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:33.082652 11563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:33.083001 11563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:33.086401 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "91a8ca99f2984301af02199cdb5d78fa"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "937800584fd6f5407867dfa6bd626351"
server_key_iv: "bfbf4f4d22b991a7f416766dd67198e6"
server_key_version: "encryptionkey@0"
I20260926 20:46:33.087339 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "91a8ca99f2984301af02199cdb5d78fa"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "937800584fd6f5407867dfa6bd626351"
server_key_iv: "bfbf4f4d22b991a7f416766dd67198e6"
server_key_version: "encryptionkey@0"
I20260926 20:46:33.093066 11563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:33.097078 11578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:33.098879 11563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 20:46:33.099097 11563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "91a8ca99f2984301af02199cdb5d78fa"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "937800584fd6f5407867dfa6bd626351"
server_key_iv: "bfbf4f4d22b991a7f416766dd67198e6"
server_key_version: "encryptionkey@0"
I20260926 20:46:33.099316 11563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:33.127229 11563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:33.131495 11563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:33.131803 11563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:33.150058 11563 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:38333
I20260926 20:46:33.150095 11629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:38333 every 8 connection(s)
I20260926 20:46:33.151500 11563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:33.152576 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11563
I20260926 20:46:33.152783 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:33.153554 5432 external_mini_cluster.cc:1428] Setting key b9522a7265fcdf6a524df58c9748497b
I20260926 20:46:33.156416 11630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:46:33 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:33.169791 11630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Bootstrap starting.
I20260926 20:46:33.173399 11630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:33.174584 11630 log.cc:824] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:33.177608 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:33.155744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.5.78.62:38333)
0926 20:46:33.156585 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:33.156597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:33.156638 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:46:33.158239 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:33.158262 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:33.158732 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:33.159107 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:33.160938 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:33.162546 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:33.164375 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:33.165080 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:33.165229 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:33.169748 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:33.169756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:33.169772 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:33.169811 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:33.173275 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:33.174070 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:33.174086 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:33.174093 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:33.174270 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:33.174701 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:33.174707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:33.174710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:33.175495 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:33.175842 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:33.176197 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 20:46:33.176593 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":352,"threads_started":1}
I20260926 20:46:33.177927 11630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: No bootstrap required, opened a new log
I20260926 20:46:33.185302 11630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } }
I20260926 20:46:33.185714 11630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:33.185842 11630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a8ca99f2984301af02199cdb5d78fa, State: Initialized, Role: FOLLOWER
I20260926 20:46:33.186444 11630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } }
I20260926 20:46:33.186712 11630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:33.186856 11630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:33.187031 11630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:33.189226 11630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } }
I20260926 20:46:33.189769 11630 leader_election.cc:302] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91a8ca99f2984301af02199cdb5d78fa; no voters:
I20260926 20:46:33.190376 11630 leader_election.cc:288] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:33.190583 11635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:33.191314 11635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 LEADER]: Becoming Leader. State: Replica: 91a8ca99f2984301af02199cdb5d78fa, State: Running, Role: LEADER
I20260926 20:46:33.191895 11635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } }
I20260926 20:46:33.192564 11630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:33.195075 11637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a8ca99f2984301af02199cdb5d78fa. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } }
I20260926 20:46:33.195525 11637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:33.195574 11636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } }
I20260926 20:46:33.196373 11636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:33.199308 11641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:33.205411 11641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:33.214893 11641 catalog_manager.cc:1347] Generated new cluster ID: 8cd0707f941e4faa8950b9baa9b5c7d2
I20260926 20:46:33.215047 11641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:33.239267 11641 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:33.240604 11641 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:33.253022 11641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Generated new TSK 0
I20260926 20:46:33.253958 11641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab.
Entry for principal oryx/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab.
I20260926 20:46:33.351595 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38333
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:33.694312 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:33.694710 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:33.694885 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:33.702849 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:33.702998 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:33.703226 11656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:33.713552 11656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:33.715046 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:33.716430 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:33.730581 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:33.730540 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:33.731432 11664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:33.731823 11656 server_base.cc:1034] running on GCE node
I20260926 20:46:33.732507 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:33.733491 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:33.734716 11656 hybrid_clock.cc:648] HybridClock initialized: now 1790455593734668 us; error 37 us; skew 500 ppm
I20260926 20:46:33.735030 11656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:33 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455593, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:33.738258 11656 init.cc:377] Logged in from keytab as oryx/127.5.78.1@KRBTEST.COM (short username oryx)
I20260926 20:46:33.739822 11656 webserver.cc:460] Webserver started at http://127.5.78.1:45501/ using document root <none> and password file <none>
I20260926 20:46:33.740576 11656 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:33.740711 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:33.741036 11656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:33.744225 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "e0f2e1e5187f475ba35eeec7369b743c"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "1543a2b2e00ddcb1c242be21d391b3e8"
server_key_iv: "ff1b4c9eb09eec3b2ac821e0f9d56503"
server_key_version: "encryptionkey@0"
I20260926 20:46:33.745062 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "e0f2e1e5187f475ba35eeec7369b743c"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "1543a2b2e00ddcb1c242be21d391b3e8"
server_key_iv: "ff1b4c9eb09eec3b2ac821e0f9d56503"
server_key_version: "encryptionkey@0"
I20260926 20:46:33.750794 11656 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:46:33.755023 11671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:33.756774 11656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:33.756979 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "e0f2e1e5187f475ba35eeec7369b743c"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "1543a2b2e00ddcb1c242be21d391b3e8"
server_key_iv: "ff1b4c9eb09eec3b2ac821e0f9d56503"
server_key_version: "encryptionkey@0"
I20260926 20:46:33.757202 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:33.789508 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:33.794608 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:33.794962 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:33.796029 11656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:33.797492 11656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:33.797581 11656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:33.797711 11656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:33.797787 11656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:33.834388 11656 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:32775
I20260926 20:46:33.834506 11783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:32775 every 8 connection(s)
I20260926 20:46:33.835979 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:33.843627 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11656
I20260926 20:46:33.843844 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:33.844543 5432 external_mini_cluster.cc:1428] Setting key 3f698898ca27f69be868940bf9bb99c2
Sep 26 20:46:33 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455593, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:33.858664 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:33.838771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:60243 (local address 127.5.78.62:38333)
0926 20:46:33.839228 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:33.839232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:33.840184 (+ 952us) server_negotiation.cc:407] Connection header received
0926 20:46:33.842088 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:33.842094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:33.842199 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:33.842323 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:33.844058 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:33.844867 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:33.846913 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:33.847594 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:33.847747 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:33.851807 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:33.851816 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:33.851824 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:33.851870 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:33.854927 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:33.855941 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:33.855945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:33.855947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:33.856068 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:33.856631 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:33.856635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:33.856637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:33.856949 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:33.857086 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:33.858105 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0926 20:46:33.858311 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":248,"threads_started":1}
I20260926 20:46:33.860175 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:33.839255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:60243)
0926 20:46:33.839993 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:33.840036 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:33.841758 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:33.842611 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:33.842620 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:33.843176 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:33.843851 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:33.843872 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:33.845029 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:33.845034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:33.846701 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:33.846722 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:33.847970 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:33.847980 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:33.848244 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:33.848966 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:33.848990 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:33.851529 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:33.855083 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:33.855096 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:33.855118 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:33.855788 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:33.856184 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:33.856191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:33.856198 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:33.856466 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:33.857091 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:33.857102 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:33.857613 (+ 511us) client_negotiation.cc:769] Sending connection context
0926 20:46:33.857971 (+ 358us) client_negotiation.cc:240] Negotiation successful
0926 20:46:33.858604 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":309,"threads_started":1}
I20260926 20:46:33.862144 11784 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38333
I20260926 20:46:33.862535 11784 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:33.863529 11784 heartbeater.cc:511] Master 127.5.78.62:38333 requested a full tablet report, sending...
I20260926 20:46:33.866456 11595 ts_manager.cc:194] Registered new tserver with Master: e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
WARNING: no policy specified for oryx/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:46:33.868604 11595 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:60243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab.
Entry for principal oryx/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab.
I20260926 20:46:33.897496 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38333
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:34.235461 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:34.235821 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:34.235996 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:34.243408 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:34.243577 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:34.243760 11790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:34.253975 11790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:34.255460 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:34.256814 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:34.271442 11796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:34.271468 11795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:34.271685 11798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:34.272490 11790 server_base.cc:1034] running on GCE node
I20260926 20:46:34.273211 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:34.274257 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:34.275512 11790 hybrid_clock.cc:648] HybridClock initialized: now 1790455594275471 us; error 63 us; skew 500 ppm
I20260926 20:46:34.275887 11790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:34 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455594, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:34.279218 11790 init.cc:377] Logged in from keytab as oryx/127.5.78.2@KRBTEST.COM (short username oryx)
I20260926 20:46:34.280745 11790 webserver.cc:460] Webserver started at http://127.5.78.2:46167/ using document root <none> and password file <none>
I20260926 20:46:34.281420 11790 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:34.281522 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:34.281785 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:34.284934 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "65dce56cb48f4dbea72fec3593fe18fe"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "4e8d3858f306d0508204b2e2fd56f8d7"
server_key_iv: "f250a46b2e1968d9c5c442cefbdbe044"
server_key_version: "encryptionkey@0"
I20260926 20:46:34.285719 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "65dce56cb48f4dbea72fec3593fe18fe"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "4e8d3858f306d0508204b2e2fd56f8d7"
server_key_iv: "f250a46b2e1968d9c5c442cefbdbe044"
server_key_version: "encryptionkey@0"
I20260926 20:46:34.291206 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:46:34.294845 11805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:34.296298 11790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:46:34.296438 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "65dce56cb48f4dbea72fec3593fe18fe"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "4e8d3858f306d0508204b2e2fd56f8d7"
server_key_iv: "f250a46b2e1968d9c5c442cefbdbe044"
server_key_version: "encryptionkey@0"
I20260926 20:46:34.296622 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:34.313172 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:34.317772 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:34.318099 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:34.319193 11790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:34.320556 11790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:34.320632 11790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:34.320719 11790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:34.320765 11790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:34.355271 11790 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:40679
I20260926 20:46:34.355376 11917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:40679 every 8 connection(s)
I20260926 20:46:34.356604 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:34.359211 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11790
I20260926 20:46:34.359388 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:34.359961 5432 external_mini_cluster.cc:1428] Setting key 64a71272d92cfa7aa82e98c8d77cd2fd
Sep 26 20:46:34 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455594, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:46:34.379060 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:34.359428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:55557 (local address 127.5.78.62:38333)
0926 20:46:34.359917 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:34.359923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:34.361185 (+ 1262us) server_negotiation.cc:407] Connection header received
0926 20:46:34.363336 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:34.363342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:34.363439 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:34.363576 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:34.365225 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.365977 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.367736 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.368214 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.368366 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:34.372615 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:34.372622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:34.372626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:34.372663 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:34.375527 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:34.376402 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:34.376408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:34.376411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:34.376551 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:34.377038 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:34.377049 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:34.377051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:34.377354 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:34.377522 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:34.378314 (+ 792us) server_negotiation.cc:299] Negotiation successful
0926 20:46:34.378607 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":247,"threads_started":1}
I20260926 20:46:34.380265 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:34.360113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:55557)
0926 20:46:34.361028 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:34.361090 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:34.363026 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:34.363832 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:34.363841 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:34.364397 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:34.365018 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:34.365038 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.366164 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.366169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:34.367567 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:34.367583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.368467 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.368477 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:34.368758 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:34.369696 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:34.369723 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:34.372328 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:34.375720 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:34.375731 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:34.375757 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:34.376236 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:34.376680 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:34.376685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:34.376690 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:34.376907 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:34.377495 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:34.377504 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:34.377912 (+ 408us) client_negotiation.cc:769] Sending connection context
0926 20:46:34.378222 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 20:46:34.378867 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":383,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.3@KRBTEST.COM" created.
I20260926 20:46:34.382072 11918 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38333
I20260926 20:46:34.382481 11918 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:34.383260 11918 heartbeater.cc:511] Master 127.5.78.62:38333 requested a full tablet report, sending...
I20260926 20:46:34.384918 11595 ts_manager.cc:194] Registered new tserver with Master: 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679)
I20260926 20:46:34.386149 11595 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:55557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab.
Entry for principal oryx/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab.
I20260926 20:46:34.409638 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38333
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:34.746236 11924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:34.746706 11924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:34.746913 11924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:34.754483 11924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:34.754609 11924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:34.754827 11924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:34.765100 11924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:34.766531 11924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:34.767903 11924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:34.783573 11929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:34.783583 11932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:34.783607 11930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:34.784308 11924 server_base.cc:1034] running on GCE node
I20260926 20:46:34.784991 11924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:34.785938 11924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:34.787139 11924 hybrid_clock.cc:648] HybridClock initialized: now 1790455594787093 us; error 39 us; skew 500 ppm
I20260926 20:46:34.787436 11924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:34 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455594, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:34.795745 11924 init.cc:377] Logged in from keytab as oryx/127.5.78.3@KRBTEST.COM (short username oryx)
I20260926 20:46:34.797253 11924 webserver.cc:460] Webserver started at http://127.5.78.3:45591/ using document root <none> and password file <none>
I20260926 20:46:34.797992 11924 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:34.798122 11924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:34.798518 11924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:34.801772 11924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "cec95cbfc877509972053ccb2917e897"
server_key_iv: "fc1cf18766d8b73841994533ae03fa70"
server_key_version: "encryptionkey@0"
I20260926 20:46:34.802628 11924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "cec95cbfc877509972053ccb2917e897"
server_key_iv: "fc1cf18766d8b73841994533ae03fa70"
server_key_version: "encryptionkey@0"
I20260926 20:46:34.808121 11924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:46:34.811599 11939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:34.812944 11924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:46:34.813105 11924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "cec95cbfc877509972053ccb2917e897"
server_key_iv: "fc1cf18766d8b73841994533ae03fa70"
server_key_version: "encryptionkey@0"
I20260926 20:46:34.813314 11924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:34.834053 11924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:34.839279 11924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:34.839583 11924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:34.840569 11924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:34.841943 11924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:34.842017 11924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:34.842177 11924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:34.842240 11924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:34.872469 11784 heartbeater.cc:503] Master 127.5.78.62:38333 was elected leader, sending a full tablet report...
I20260926 20:46:34.876215 11924 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38237
I20260926 20:46:34.876365 12051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38237 every 8 connection(s)
I20260926 20:46:34.877482 11924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:34.883136 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 11924
I20260926 20:46:34.883296 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:34.883718 5432 external_mini_cluster.cc:1428] Setting key e4e37695e25d7ab3582f16e1033dc2bd
Sep 26 20:46:34 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455594, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:34.897501 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:34.880147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:36123 (local address 127.5.78.62:38333)
0926 20:46:34.880545 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:34.880549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:34.881606 (+ 1057us) server_negotiation.cc:407] Connection header received
0926 20:46:34.883563 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:34.883568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:34.883658 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:34.883784 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:34.885459 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.886233 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.887982 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.888397 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.888538 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:34.891967 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:34.891972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:34.891975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:34.892006 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:34.894481 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:34.895180 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:34.895186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:34.895189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:34.895308 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:34.895748 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:34.895753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:34.895756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:34.896015 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:34.896169 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:34.896882 (+ 713us) server_negotiation.cc:299] Negotiation successful
0926 20:46:34.897106 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":191,"threads_started":1}
I20260926 20:46:34.898690 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:34.880765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:36123)
0926 20:46:34.881446 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:34.881483 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:34.883253 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:34.884064 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:34.884073 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:34.884621 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:34.885295 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:34.885313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.886393 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.886443 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:34.887832 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:34.887844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:34.888632 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:34.888640 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:34.888874 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:34.889531 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:34.889557 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:34.891758 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:34.894613 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:34.894621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:34.894640 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:34.895052 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:34.895425 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:34.895430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:34.895434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:34.895624 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:34.896142 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:34.896150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:34.896493 (+ 343us) client_negotiation.cc:769] Sending connection context
0926 20:46:34.896762 (+ 269us) client_negotiation.cc:240] Negotiation successful
0926 20:46:34.897304 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":269,"threads_started":1}
I20260926 20:46:34.900218 12052 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38333
I20260926 20:46:34.900532 12052 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:34.901311 12052 heartbeater.cc:511] Master 127.5.78.62:38333 requested a full tablet report, sending...
I20260926 20:46:34.902791 11595 ts_manager.cc:194] Registered new tserver with Master: 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237)
I20260926 20:46:34.903787 11595 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:36123
I20260926 20:46:34.912053 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.78.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.61.keytab.
Entry for principal oryx/127.5.78.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.61.keytab.
I20260926 20:46:34.958514 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:45599
--webserver_interface=127.5.78.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.61.keytab
--principal=oryx/127.5.78.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:38333,127.5.78.61:45599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:35.296104 12058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:35.296419 12058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:35.296578 12058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:35.304342 12058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:35.304553 12058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:35.304632 12058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:35.304685 12058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:35.304721 12058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:35.315102 12058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:38333,127.5.78.61:45599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.61.keytab
--principal=oryx/127.5.78.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:45599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.78.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
W20260926 20:46:35.315461 12058 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.78.62:38333,127.5.78.61:45599'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 20:46:35.316599 12058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:35.318161 12058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:35.331301 12063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:35.331383 12066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:35.331344 12064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:35.332471 12058 server_base.cc:1034] running on GCE node
I20260926 20:46:35.333439 12058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:35.335176 12058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:35.336491 12058 hybrid_clock.cc:648] HybridClock initialized: now 1790455595336437 us; error 63 us; skew 500 ppm
I20260926 20:46:35.336894 12058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:35 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455595, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:35.340430 12058 init.cc:377] Logged in from keytab as oryx/127.5.78.61@KRBTEST.COM (short username oryx)
I20260926 20:46:35.342155 12058 webserver.cc:460] Webserver started at http://127.5.78.61:40031/ using document root <none> and password file <none>
I20260926 20:46:35.342957 12058 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:35.343086 12058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:35.343400 12058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:35.346463 12058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/instance:
uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
format_stamp: "Formatted at 2026-09-26 20:46:35 on dist-test-slave-z7j8"
server_key: "3f49006edbb40c045abb15163901c8c9"
server_key_iv: "f961408bd9485a75e0e88419d9d307d8"
server_key_version: "encryptionkey@0"
I20260926 20:46:35.347306 12058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal/instance:
uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
format_stamp: "Formatted at 2026-09-26 20:46:35 on dist-test-slave-z7j8"
server_key: "3f49006edbb40c045abb15163901c8c9"
server_key_iv: "f961408bd9485a75e0e88419d9d307d8"
server_key_version: "encryptionkey@0"
I20260926 20:46:35.353029 12058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:35.357673 12073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:35.359694 12058 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:46:35.359901 12058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
format_stamp: "Formatted at 2026-09-26 20:46:35 on dist-test-slave-z7j8"
server_key: "3f49006edbb40c045abb15163901c8c9"
server_key_iv: "f961408bd9485a75e0e88419d9d307d8"
server_key_version: "encryptionkey@0"
I20260926 20:46:35.360153 12058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:35.375344 12058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:35.380045 12058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:35.380281 12058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:35.389109 11918 heartbeater.cc:503] Master 127.5.78.62:38333 was elected leader, sending a full tablet report...
I20260926 20:46:35.399571 12058 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.61:45599
I20260926 20:46:35.399606 12124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.61:45599 every 8 connection(s)
I20260926 20:46:35.401063 12058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
I20260926 20:46:35.404906 12125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:35.409744 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 12058
I20260926 20:46:35.411966 12125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:46:35 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455595, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.61@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
Sep 26 20:46:35 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
I20260926 20:46:35.433316 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.414969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.5.78.62:38333)
0926 20:46:35.415517 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.415525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.416149 (+ 624us) server_negotiation.cc:407] Connection header received
0926 20:46:35.416218 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.416224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.416295 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.416406 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.418060 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.419210 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.421351 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.421976 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.422124 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.426532 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.426541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.426548 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.426595 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.429487 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.430525 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.430529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.430531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.430681 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.431173 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.431181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.431184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.431621 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.431757 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.432740 (+ 983us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.432961 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":256,"threads_started":1}
I20260926 20:46:35.434545 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.414953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:38752)
0926 20:46:35.415851 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:35.415897 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:35.416069 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:35.416501 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:35.416509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:35.417393 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:35.417903 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.417922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.419356 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.419359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.421210 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.421223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.422367 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.422375 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.422569 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.423335 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:35.423369 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:35.426236 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:35.429629 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.429639 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.429652 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.430371 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.430820 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.430825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.430829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.431040 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.431893 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:35.431903 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:35.432366 (+ 463us) client_negotiation.cc:769] Sending connection context
0926 20:46:35.432663 (+ 297us) client_negotiation.cc:240] Negotiation successful
0926 20:46:35.433199 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":502,"threads_started":1}
I20260926 20:46:35.434542 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.411908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.5.78.61:45599)
0926 20:46:35.412752 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.412764 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.412815 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:46:35.414880 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.414908 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.415515 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.415936 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.417777 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.419524 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.421240 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.422065 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.422265 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.427448 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.427460 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.427479 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.427518 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.430875 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.431712 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.431721 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.431724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.431899 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.432348 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.432352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.432354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.432713 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.433076 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.433823 (+ 747us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.434013 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":366,"threads_started":1}
I20260926 20:46:35.436589 12125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:46:35 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455595, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.61@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
I20260926 20:46:35.449249 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.437432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:53994)
0926 20:46:35.437615 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:35.437631 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:35.437717 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:35.438062 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:35.438065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:35.438459 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:35.438608 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.438616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.439539 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.439542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.440403 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.440412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.441020 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.441024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.441168 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.441771 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:35.441790 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:35.444031 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:35.447075 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.447079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.447083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.447481 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.447823 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.447826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.447828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.447951 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.448455 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:35.448459 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:35.448588 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 20:46:35.448726 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 20:46:35.448952 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 20:46:35.449384 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.437686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53994 (local address 127.5.78.61:45599)
0926 20:46:35.437840 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.437844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.437859 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:46:35.437897 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.437900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.437950 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.438027 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.438736 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.439435 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.440519 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.440903 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.441000 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.444226 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.444231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.444234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.444263 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.446942 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.447606 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.447610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.447612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.447725 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.448043 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.448047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.448055 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.448330 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.448455 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.448801 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.449087 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":45}
I20260926 20:46:35.454186 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Bootstrap starting.
I20260926 20:46:35.457446 12125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:35.458572 12125 log.cc:824] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:35.461421 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: No bootstrap required, opened a new log
I20260926 20:46:35.469718 12125 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } }
I20260926 20:46:35.470103 12125 raft_consensus.cc:374] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:35.470223 12125 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bebec0aad9d04aed8d8d8f7d6ed48653, State: Initialized, Role: FOLLOWER
I20260926 20:46:35.470888 12125 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } }
I20260926 20:46:35.471705 12132 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } } }
I20260926 20:46:35.471989 12132 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:46:35.472714 12125 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:46:35.482053 12143 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:46:35.482234 12143 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:46:35.484522 12058 master_runner.cc:428] Detected that this master bebec0aad9d04aed8d8d8f7d6ed48653 is joining an existing cluster
I20260926 20:46:35.484661 12058 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.78.61:45599
I20260926 20:46:35.512382 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.489690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:54002)
0926 20:46:35.493610 (+ 3920us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:35.493640 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:35.494280 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:35.494924 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:35.494932 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:35.495806 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:35.496345 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.496363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.498386 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.498394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.500718 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.500736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.502153 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.502159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.502399 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.503136 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:35.503162 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:35.504457 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:35.508949 (+ 4492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.508954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.508958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.509451 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.510470 (+ 1019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.510478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.510484 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.510646 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.511313 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:35.511318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:35.511473 (+ 155us) client_negotiation.cc:769] Sending connection context
0926 20:46:35.511600 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 20:46:35.511895 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3696,"thread_start_us":9018,"threads_started":1}
I20260926 20:46:35.513309 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.489144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.5.78.61:45599)
0926 20:46:35.489338 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.489344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.493761 (+ 4417us) server_negotiation.cc:407] Connection header received
0926 20:46:35.494476 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.494485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.494583 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.494668 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.496560 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.498098 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.500894 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.501686 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.501787 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.504655 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.504662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.504667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.504709 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.508807 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.509570 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.509576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.509579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.509723 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.510766 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.510771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.510774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.511107 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.511265 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.512758 (+ 1493us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.512957 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 20:46:35.517154 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.489727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:38764)
0926 20:46:35.499075 (+ 9348us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:35.499096 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:35.499281 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:35.499817 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:35.499824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:35.501094 (+ 1270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:35.501446 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.501460 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.503035 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.503043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.505078 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.505097 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.506467 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.506477 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.506713 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.507332 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:35.507359 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:35.508723 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:35.513283 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.513295 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.513303 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.514687 (+ 1384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.515140 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.515144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.515148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.515320 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.516025 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:35.516030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:35.516210 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 20:46:35.516362 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 20:46:35.516752 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9204,"thread_start_us":4051,"threads_started":1}
WARNING: no policy specified for oryx/127.5.78.60@KRBTEST.COM; defaulting to no policy
I20260926 20:46:35.517202 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.489205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38764 (local address 127.5.78.62:38333)
0926 20:46:35.489357 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.489364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.499181 (+ 9817us) server_negotiation.cc:407] Connection header received
0926 20:46:35.499433 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.499439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.499531 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.499599 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.501633 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.502815 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.505261 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.506031 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.506141 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.508947 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.508957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.508964 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.509009 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.513120 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.514828 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.514836 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.514841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.514992 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.515440 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.515447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.515453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.515833 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.516076 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.516512 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.516777 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.60@KRBTEST.COM" created.
Sep 26 20:46:35 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455593, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.62@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
I20260926 20:46:35.544970 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.526570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.5.78.61:45599)
0926 20:46:35.526756 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.526762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.526781 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:46:35.526878 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.526883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.526978 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.527083 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.528322 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.529401 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.531528 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.532255 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.532395 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.536880 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.536888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.536893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.536942 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.541208 (+ 4266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.542538 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.542543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.542545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.542701 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.543244 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.543249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.543251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.543595 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.543821 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.544322 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.544581 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:46:35.545102 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.523689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:54010)
0926 20:46:35.525229 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:35.525285 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:35.525452 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:35.527169 (+ 1717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:35.527179 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:35.527958 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:35.528170 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.528180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.529547 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.529556 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.531372 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.531389 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.532440 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.532448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.532696 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.533526 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:35.533553 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:35.536621 (+ 3068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:35.541411 (+ 4790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.541430 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.541441 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.542346 (+ 905us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.542850 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.542855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.542859 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.543060 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.543821 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:35.543831 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:35.544020 (+ 189us) client_negotiation.cc:769] Sending connection context
0926 20:46:35.544213 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 20:46:35.544578 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"thread_start_us":250,"threads_started":1}
I20260926 20:46:35.547396 11595 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.78.61:45599: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
member_type: NON_VOTER
last_known_addr {
host: "127.5.78.61"
port: 45599
}
attrs {
promote: true
}
}
dest_uuid: "91a8ca99f2984301af02199cdb5d78fa"
cas_config_opid_index: -1
I20260926 20:46:35.548671 11595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.60.keytab.
Entry for principal oryx/127.5.78.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.60.keytab.
I20260926 20:46:35.550740 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:46291
--webserver_interface=127.5.78.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.60.keytab
--principal=oryx/127.5.78.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:46:35.561542 12100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:35.561124 12156 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bebec0aad9d04aed8d8d8f7d6ed48653 (127.5.78.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } }
I20260926 20:46:35.566042 12156 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.78.61:45599
I20260926 20:46:35.566179 12100 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Refusing update from remote peer 91a8ca99f2984301af02199cdb5d78fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 20:46:35.566992 12058 master.cc:537] Master@127.5.78.61:45599 shutting down...
I20260926 20:46:35.567744 12155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } }
I20260926 20:46:35.568198 12155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:35.568611 12156 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 20:46:35.578023 11582 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 20:46:35.584381 11582 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa -> Peer bebec0aad9d04aed8d8d8f7d6ed48653 (127.5.78.61:45599): Couldn't send request to peer bebec0aad9d04aed8d8d8f7d6ed48653. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 20:46:35.584591 12058 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:35.585057 12058 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:35.585247 12058 tablet_replica.cc:330] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: stopping tablet replica
I20260926 20:46:35.592523 12058 master.cc:552] Master@127.5.78.61:45599 shutdown complete.
I20260926 20:46:35.592625 12058 master_runner.cc:305] Clearing existing system tablet
I20260926 20:46:35.593155 12058 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:35.593254 12058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:35.596515 12058 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260926 20:46:35.598340 12162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:35.598800 12058 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:35.599225 12058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 20:46:35.599357 12058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
format_stamp: "Formatted at 2026-09-26 20:46:35 on dist-test-slave-z7j8"
server_key: "3f49006edbb40c045abb15163901c8c9"
server_key_iv: "f961408bd9485a75e0e88419d9d307d8"
server_key_version: "encryptionkey@0"
I20260926 20:46:35.604804 12058 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:35.612464 12058 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 20:46:35.612811 12058 log.cc:1196] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 20:46:35.613346 12058 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Deleting consensus metadata
I20260926 20:46:35.613694 12058 master_runner.cc:315] Copying system tablet from 127.5.78.62:38333
I20260926 20:46:35.616770 12058 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: tablet copy: Beginning tablet copy session from remote peer at address 127.5.78.62:38333
I20260926 20:46:35.638213 12168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.617831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:38774)
0926 20:46:35.622985 (+ 5154us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:35.623010 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:35.623195 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:35.623633 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:35.623639 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:35.624175 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:35.624449 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.624464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.625991 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.625998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.627519 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:35.627536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.628633 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.628641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:35.628836 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.629650 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:35.629673 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:35.630881 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:35.635146 (+ 4265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.635153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.635158 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.635690 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.636188 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:35.636194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:35.636198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:35.636380 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:35.637074 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:35.637080 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:35.637257 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 20:46:35.637411 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 20:46:35.637760 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4995,"thread_start_us":4876,"threads_started":1}
I20260926 20:46:35.638279 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:35.617806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.5.78.62:38333)
0926 20:46:35.617996 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:35.618002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:35.623160 (+ 5158us) server_negotiation.cc:407] Connection header received
0926 20:46:35.623338 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:35.623345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:35.623456 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:35.623593 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:35.624655 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.625814 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.627693 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:35.628445 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:35.628604 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:35.631152 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:35.631160 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:35.631165 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:35.631213 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:35.634944 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.635860 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.635867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.635871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.636055 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:35.636520 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:35.636526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:35.636530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:35.636939 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:35.637107 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:35.637592 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 20:46:35.637834 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:46:35.639814 11615 tablet_copy_service.cc:140] P 91a8ca99f2984301af02199cdb5d78fa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bebec0aad9d04aed8d8d8f7d6ed48653 ({username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:38774)
I20260926 20:46:35.640126 11615 tablet_copy_service.cc:161] P 91a8ca99f2984301af02199cdb5d78fa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bebec0aad9d04aed8d8d8f7d6ed48653 at {username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:38774: session id = bebec0aad9d04aed8d8d8f7d6ed48653-00000000000000000000000000000000
I20260926 20:46:35.645084 11615 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 20:46:35.647742 12058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 20:46:35.655052 12058 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: tablet copy: Starting download of 0 data blocks...
I20260926 20:46:35.655416 12058 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: tablet copy: Starting download of 1 WAL segments...
I20260926 20:46:35.658670 12058 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 20:46:35.663895 11615 tablet_copy_service.cc:342] P 91a8ca99f2984301af02199cdb5d78fa: Request end of tablet copy session bebec0aad9d04aed8d8d8f7d6ed48653-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:38774
I20260926 20:46:35.664160 11615 tablet_copy_service.cc:433] P 91a8ca99f2984301af02199cdb5d78fa: ending tablet copy session bebec0aad9d04aed8d8d8f7d6ed48653-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bebec0aad9d04aed8d8d8f7d6ed48653
I20260926 20:46:35.678139 12058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:35.681478 12169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:35.686690 12170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:35.688464 12172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:35.689936 12058 server_base.cc:1034] running on GCE node
I20260926 20:46:35.690374 12058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:35.691032 12058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:35.692286 12058 hybrid_clock.cc:648] HybridClock initialized: now 1790455595692191 us; error 52 us; skew 500 ppm
I20260926 20:46:35.692555 12058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:35 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455595, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:35.694666 12058 init.cc:377] Logged in from keytab as oryx/127.5.78.61@KRBTEST.COM (short username oryx)
I20260926 20:46:35.696007 12058 webserver.cc:460] Webserver started at http://127.5.78.61:45951/ using document root <none> and password file <none>
I20260926 20:46:35.696359 12058 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:35.696452 12058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:35.698828 12058 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 20:46:35.700729 12179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:35.701180 12058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 20:46:35.701352 12058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
format_stamp: "Formatted at 2026-09-26 20:46:35 on dist-test-slave-z7j8"
server_key: "3f49006edbb40c045abb15163901c8c9"
server_key_iv: "f961408bd9485a75e0e88419d9d307d8"
server_key_version: "encryptionkey@0"
I20260926 20:46:35.701526 12058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:35.727092 12058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:35.730819 12058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:35.755172 12058 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.61:45599
I20260926 20:46:35.755198 12230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.61:45599 every 8 connection(s)
I20260926 20:46:35.756098 12058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
I20260926 20:46:35.763628 12231 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:46:35.768572 12231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Bootstrap starting.
I20260926 20:46:35.792528 12231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:46:35.793190 12231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Bootstrap complete.
I20260926 20:46:35.794543 12231 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } }
I20260926 20:46:35.794853 12231 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bebec0aad9d04aed8d8d8f7d6ed48653, State: Initialized, Role: LEARNER
I20260926 20:46:35.795178 12231 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } }
I20260926 20:46:35.798913 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } }
I20260926 20:46:35.799427 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: LEARNER
I20260926 20:46:35.799489 12231 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:35.813700 12245 catalog_manager.cc:1259] Loaded cluster ID: 8cd0707f941e4faa8950b9baa9b5c7d2
I20260926 20:46:35.813853 12245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: loading cluster ID for follower catalog manager: success
I20260926 20:46:35.817981 12245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: acquiring CA information for follower catalog manager: success
I20260926 20:46:35.820835 12245 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:46:35.906311 12052 heartbeater.cc:503] Master 127.5.78.62:38333 was elected leader, sending a full tablet report...
W20260926 20:46:35.980640 12157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:35.981050 12157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:35.981242 12157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:35.989122 12157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:35.989315 12157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:35.989446 12157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:35.989563 12157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:35.989657 12157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:36.000641 12157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.60.keytab
--principal=oryx/127.5.78.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:46291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.78.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:36.002390 12157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:36.003983 12157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:36.016109 12250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:36.016086 12251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:36.016413 12253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:36.016891 12157 server_base.cc:1034] running on GCE node
I20260926 20:46:36.017737 12157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:36.019238 12157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:36.020479 12157 hybrid_clock.cc:648] HybridClock initialized: now 1790455596020429 us; error 41 us; skew 500 ppm
I20260926 20:46:36.020838 12157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:36.024183 12157 init.cc:377] Logged in from keytab as oryx/127.5.78.60@KRBTEST.COM (short username oryx)
I20260926 20:46:36.025673 12157 webserver.cc:460] Webserver started at http://127.5.78.60:41161/ using document root <none> and password file <none>
I20260926 20:46:36.026566 12157 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:36.026726 12157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:36.027045 12157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:36.030010 12157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/instance:
uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
format_stamp: "Formatted at 2026-09-26 20:46:36 on dist-test-slave-z7j8"
server_key: "8c162798692c8ed85dd39bf269f6d737"
server_key_iv: "acf6a505c0fe29271511df2d7b6b9f3b"
server_key_version: "encryptionkey@0"
I20260926 20:46:36.030833 12157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal/instance:
uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
format_stamp: "Formatted at 2026-09-26 20:46:36 on dist-test-slave-z7j8"
server_key: "8c162798692c8ed85dd39bf269f6d737"
server_key_iv: "acf6a505c0fe29271511df2d7b6b9f3b"
server_key_version: "encryptionkey@0"
I20260926 20:46:36.035940 12157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 20:46:36.039397 12260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:36.040804 12157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:46:36.041006 12157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
format_stamp: "Formatted at 2026-09-26 20:46:36 on dist-test-slave-z7j8"
server_key: "8c162798692c8ed85dd39bf269f6d737"
server_key_iv: "acf6a505c0fe29271511df2d7b6b9f3b"
server_key_version: "encryptionkey@0"
I20260926 20:46:36.041240 12157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:36.053529 12157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:36.058100 12157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:36.058451 12157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:36.077601 12157 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.60:46291
I20260926 20:46:36.077611 12311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.60:46291 every 8 connection(s)
I20260926 20:46:36.079113 12157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
I20260926 20:46:36.082872 12312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:36.089483 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 12157
I20260926 20:46:36.089536 12312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.60@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.60@KRBTEST.COM
I20260926 20:46:36.110455 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.092458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.5.78.62:38333)
0926 20:46:36.092639 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.092645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.093442 (+ 797us) server_negotiation.cc:407] Connection header received
0926 20:46:36.094887 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.094892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.094955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.095074 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.096414 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.097138 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.098955 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.099584 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.099678 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.103844 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.103849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.103852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.103883 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.107010 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.107881 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.107886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.107890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.108024 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.108468 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.108472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.108477 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.108742 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.108892 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.109851 (+ 959us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.110088 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:46:36.112757 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.091558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.5.78.60:46291)
0926 20:46:36.092376 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.092386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.092424 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:46:36.094582 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.094612 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.094910 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.095123 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.096545 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.097975 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.099530 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.100464 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.100612 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.105807 (+ 5195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.105819 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.105835 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.105875 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.109062 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.110016 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.110025 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.110028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.110174 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.110653 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.110660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.110666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.111092 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.111339 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.111931 (+ 592us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.112149 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":586,"thread_start_us":372,"threads_started":1}
I20260926 20:46:36.112792 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.092437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:38780)
0926 20:46:36.093214 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.093270 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.094582 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.095125 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.095133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.095882 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.096241 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.096258 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.097272 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.097277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.098819 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.098831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.099861 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.099871 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.100216 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.101137 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.101159 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.103646 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.107156 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.107168 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.107182 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.107754 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.108130 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.108136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.108144 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.108337 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.108877 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.108884 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.109287 (+ 403us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.110796 (+ 1509us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.111317 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":23,"thread_start_us":360,"threads_started":1}
I20260926 20:46:36.114806 12312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:46:36.129860 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.111337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:54012)
0926 20:46:36.122792 (+ 11455us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.122812 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.122932 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.123372 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.123375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.123394 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:36.123553 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.123560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.125977 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.125981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.127732 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.127746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.129032 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.129040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.129235 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.129303 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.129416 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.129547 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11350,"thread_start_us":11223,"threads_started":1}
I20260926 20:46:36.130250 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.111505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.5.78.61:45599)
0926 20:46:36.122098 (+ 10593us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.122107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.122886 (+ 779us) server_negotiation.cc:407] Connection header received
0926 20:46:36.123051 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.123059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.123190 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.123349 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:36.123702 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.125776 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.127906 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.128783 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.129335 (+ 552us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.129586 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.129637 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.129763 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":10413,"thread_start_us":15087,"threads_started":1}
I20260926 20:46:36.131253 12206 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260926 20:46:36.131785 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a8ca99f2984301af02199cdb5d78fa. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } }
I20260926 20:46:36.132077 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: LEARNER
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.60@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
I20260926 20:46:36.139219 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.116446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:54018)
0926 20:46:36.116665 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.116684 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.116778 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.127206 (+ 10428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.127210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.127551 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.127689 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.127697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.128884 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.128892 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.129892 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.129903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.130855 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.130861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.131075 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.131707 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.131724 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.133830 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.136788 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.136793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.136797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.137178 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.137566 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.137570 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.137574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.137704 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.138193 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.138198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.138332 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.138563 (+ 231us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.138885 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 20:46:36.139257 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.116538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.5.78.61:45599)
0926 20:46:36.126887 (+ 10349us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.126894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.126914 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:46:36.126963 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.126968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.127037 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.127147 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.127854 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.128740 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.130088 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.130703 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.130844 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.134038 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.134043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.134046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.134075 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.136623 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.137299 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.137306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.137310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.137449 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.137809 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.137816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.137819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.138072 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.138218 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.138647 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.138862 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10224,"thread_start_us":5192,"threads_started":1}
I20260926 20:46:36.140339 12312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.60@KRBTEST.COM for oryx/127.5.78.60@KRBTEST.COM
I20260926 20:46:36.153463 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.141148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33238 (local address 127.5.78.60:46291)
0926 20:46:36.141300 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.141304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.141317 (+ 13us) server_negotiation.cc:407] Connection header received
0926 20:46:36.141356 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.141360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.141408 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.141484 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.142202 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.142983 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.144104 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.144526 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.144660 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.148052 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.148058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.148061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.148090 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.150657 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.151430 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.151434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.151437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.151602 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.152031 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.152035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.152037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.152334 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.152453 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.152879 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.153187 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":58}
I20260926 20:46:36.153486 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.140994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33238)
0926 20:46:36.141172 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.141188 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.141267 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.141585 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.141589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.141920 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.142057 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.142064 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.143108 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.143112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.143960 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.143970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.144707 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.144711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.144904 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.145672 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.145690 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.147846 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.150792 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.150798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.150803 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.151283 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.151722 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.151726 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.151731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.151918 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.152458 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.152463 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.152615 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.152755 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.153036 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 20:46:36.158392 12312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Bootstrap starting.
I20260926 20:46:36.161736 12312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:36.162907 12312 log.cc:824] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:36.165520 12312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: No bootstrap required, opened a new log
I20260926 20:46:36.172961 12312 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } }
I20260926 20:46:36.173321 12312 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:36.173441 12312 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd329c86b4f84d6b8a67f0b0ac80fbcb, State: Initialized, Role: FOLLOWER
I20260926 20:46:36.174000 12312 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } }
I20260926 20:46:36.174955 12322 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } } }
I20260926 20:46:36.175277 12322 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:46:36.175987 12312 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:46:36.185500 12333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:46:36.185657 12333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:46:36.189881 12157 master_runner.cc:428] Detected that this master bd329c86b4f84d6b8a67f0b0ac80fbcb is joining an existing cluster
I20260926 20:46:36.190052 12157 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.78.60:46291
I20260926 20:46:36.212826 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.193958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.5.78.62:38333)
0926 20:46:36.194160 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.194169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.195104 (+ 935us) server_negotiation.cc:407] Connection header received
0926 20:46:36.195413 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.195422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.195523 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.195610 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.197636 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.199188 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.201841 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.202552 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.202666 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.205716 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.205723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.205727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.205767 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.209305 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.210315 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.210320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.210323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.210523 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.211004 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.211014 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.211019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.211388 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.211506 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.212184 (+ 678us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.212435 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:46:36.214262 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.194004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:38796)
0926 20:46:36.194975 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.195002 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.195243 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.196087 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.196095 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.196920 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.197413 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.197430 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.199408 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.199417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.201693 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.201709 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.203055 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.203067 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.203454 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.204246 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.204267 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.205519 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.209471 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.209475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.209484 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.210185 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.210675 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.210681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.210684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.210861 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.211602 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.211609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.211825 (+ 216us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.213406 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.213739 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":21,"thread_start_us":625,"threads_started":1}
I20260926 20:46:36.217088 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.194055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.5.78.61:45599)
0926 20:46:36.194343 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.194349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.201627 (+ 7278us) server_negotiation.cc:407] Connection header received
0926 20:46:36.201852 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.201858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.201936 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.202002 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.203575 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.204711 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.206831 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.207511 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.207638 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.210353 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.210358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.210365 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.210428 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.213803 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.214692 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.214696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.214700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.214857 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.215301 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.215309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.215315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.215673 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.215812 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.216440 (+ 628us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.216678 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260926 20:46:36.218451 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.194601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:54024)
0926 20:46:36.201527 (+ 6926us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.201546 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.201715 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.202242 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.202246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.202989 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.203398 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.203410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.204907 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.204911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.206673 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.206689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.208122 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.208131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.208305 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.208915 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.208936 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.210147 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.213968 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.213973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.213977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.214548 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.214992 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.214997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.215003 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.215181 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.215890 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.215896 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.216159 (+ 263us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.217546 (+ 1387us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.217881 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6789,"mutex_wait_us":1,"thread_start_us":6469,"threads_started":1}
I20260926 20:46:36.220696 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11656
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455593, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.62@KRBTEST.COM for oryx/127.5.78.60@KRBTEST.COM
I20260926 20:46:36.232725 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.195383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.5.78.60:46291)
0926 20:46:36.195917 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.195929 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.214846 (+ 18917us) server_negotiation.cc:407] Connection header received
0926 20:46:36.215058 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.215068 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.215137 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.215197 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.216209 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.217306 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.221921 (+ 4615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.222716 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.222828 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.225874 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.225880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.225884 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.225915 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.229590 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.230360 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.230366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.230370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.230556 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.230947 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.230951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.230954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.231327 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.231447 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.232073 (+ 626us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.232304 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260926 20:46:36.233744 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.195122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33254)
0926 20:46:36.214769 (+ 19647us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.214790 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.214927 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.215391 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.215395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.215848 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.216067 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.216077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.219385 (+ 3308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.219390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.221787 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.221799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.223099 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.223104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.223282 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.223914 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.223936 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.225692 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.229734 (+ 4042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.229738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.229742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.230234 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.230674 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.230678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.230681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.230826 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.231549 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.231556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.231764 (+ 208us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.233005 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.233361 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19533,"thread_start_us":19456,"threads_started":1}
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.1@KRBTEST.COM
I20260926 20:46:36.238310 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.220407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33264)
0926 20:46:36.220631 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.220649 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.220744 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.221692 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.221697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.222146 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.222315 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.222326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.225069 (+ 2743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.225075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.226474 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.226489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.227876 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.227882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.228076 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.228663 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.228685 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.231335 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.235616 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.235621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.235625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.236154 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.236578 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.236585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.236590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.236737 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.237384 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.237389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.237565 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.237689 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.237959 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 20:46:36.239162 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.220753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.5.78.60:46291)
0926 20:46:36.221324 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.221331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.221356 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:46:36.221434 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.221439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.221519 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.222648 (+ 1129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.222944 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.224899 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.226929 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.227699 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.231500 (+ 3801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.231763 (+ 263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.231770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.231774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.231805 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.235467 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.236318 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.236323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.236326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.236457 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.236881 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.236886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.236889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.237244 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.238348 (+ 1104us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.238529 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.238739 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":454,"thread_start_us":314,"threads_started":1}
I20260926 20:46:36.240906 11595 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.78.60:46291: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
member_type: NON_VOTER
last_known_addr {
host: "127.5.78.60"
port: 46291
}
attrs {
promote: true
}
}
dest_uuid: "91a8ca99f2984301af02199cdb5d78fa"
cas_config_opid_index: 5
I20260926 20:46:36.242010 11595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } }
I20260926 20:46:36.243250 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.222202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43890 (local address 127.5.78.1:32775)
0926 20:46:36.224229 (+ 2027us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.224236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.224260 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:46:36.224309 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.224314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.224534 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.224725 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.226289 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.227992 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.229943 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.230964 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.234587 (+ 3623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.236447 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.236459 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.236474 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.236538 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.240057 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.240781 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.240790 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.240794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.240946 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.241377 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.241382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.241385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.241799 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.242073 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.242631 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.242892 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1875,"thread_start_us":1728,"threads_started":1}
I20260926 20:46:36.244086 12317 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.244935 11779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:36.248011 12317 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.78.60:46291
I20260926 20:46:36.248917 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 LEARNER]: Refusing update from remote peer 91a8ca99f2984301af02199cdb5d78fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 20:46:36.249815 12157 master.cc:537] Master@127.5.78.60:46291 shutting down...
W20260926 20:46:36.257332 11580 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa -> Peer bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60:46291): Couldn't send request to peer bd329c86b4f84d6b8a67f0b0ac80fbcb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 20:46:36.262944 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.263275 12157 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:36.263533 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:36.263675 12157 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:36.263901 12157 tablet_replica.cc:330] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: stopping tablet replica
I20260926 20:46:36.271730 12348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260926 20:46:36.273346 12157 master.cc:552] Master@127.5.78.60:46291 shutdown complete.
I20260926 20:46:36.273442 12157 master_runner.cc:305] Clearing existing system tablet
I20260926 20:46:36.273834 12157 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:36.273929 12157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:36.274883 12206 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.278694 12157 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 20:46:36.279168 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: NON_VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: true } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.279474 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: LEARNER
I20260926 20:46:36.280452 12344 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: attempting to promote NON_VOTER bebec0aad9d04aed8d8d8f7d6ed48653 to VOTER
I20260926 20:46:36.281515 12344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } }
I20260926 20:46:36.283957 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.283418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33266)
0926 20:46:36.283629 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.283773 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 20:46:36.284384 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 LEARNER]: Refusing update from remote peer 91a8ca99f2984301af02199cdb5d78fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 20:46:36.285143 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.284707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33276)
0926 20:46:36.284917 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.285012 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 20:46:36.285393 12352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:36.285338 12345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 20:46:36.285784 12157 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260926 20:46:36.286008 11580 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa -> Peer bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60:46291): Couldn't send request to peer bd329c86b4f84d6b8a67f0b0ac80fbcb. Status: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46291: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 20:46:36.286255 12157 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.003s
I20260926 20:46:36.286550 12157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
format_stamp: "Formatted at 2026-09-26 20:46:36 on dist-test-slave-z7j8"
server_key: "8c162798692c8ed85dd39bf269f6d737"
server_key_iv: "acf6a505c0fe29271511df2d7b6b9f3b"
server_key_version: "encryptionkey@0"
I20260926 20:46:36.287835 12344 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bebec0aad9d04aed8d8d8f7d6ed48653 (127.5.78.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.288476 12206 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bebec0aad9d04aed8d8d8f7d6ed48653 (127.5.78.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.291765 12348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.291930 12157 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:36.292092 12348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:36.292351 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.292587 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:46:36.293190 12348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.293408 12348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:36.297425 12157 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 20:46:36.297605 12157 log.cc:1196] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 20:46:36.298022 12157 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Deleting consensus metadata
I20260926 20:46:36.298259 12157 master_runner.cc:315] Copying system tablet from 127.5.78.62:38333
I20260926 20:46:36.300994 12157 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: tablet copy: Beginning tablet copy session from remote peer at address 127.5.78.62:38333
I20260926 20:46:36.318341 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.301774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38802 (local address 127.5.78.62:38333)
0926 20:46:36.302027 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.302036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.306997 (+ 4961us) server_negotiation.cc:407] Connection header received
0926 20:46:36.307242 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.307248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.307343 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.307458 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:36.308339 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.309162 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.310572 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.310989 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.311113 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.312840 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:36.312846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:36.312851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:36.312890 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:36.315605 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.316320 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.316327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.316331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.316474 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:36.316876 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:36.316884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:36.316887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:36.317158 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:36.317318 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.317763 (+ 445us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.317978 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 20:46:36.318368 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.301773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:38802)
0926 20:46:36.306796 (+ 5023us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.306890 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.307089 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.307515 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.307519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.307905 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:36.308187 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.308198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.309306 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.309312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.310382 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.310393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.311140 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.311147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.311321 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.311961 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:36.311984 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:36.312645 (+ 661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:36.315744 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.315750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.315754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.316105 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.316626 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:36.316630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:36.316632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:36.316771 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:36.317326 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:36.317333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:36.317481 (+ 148us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.317621 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.317962 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4900,"thread_start_us":4795,"threads_started":1}
I20260926 20:46:36.319249 11615 tablet_copy_service.cc:140] P 91a8ca99f2984301af02199cdb5d78fa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bd329c86b4f84d6b8a67f0b0ac80fbcb ({username='oryx', principal='oryx/127.5.78.60@KRBTEST.COM'} at 127.0.0.1:38802)
I20260926 20:46:36.319420 11615 tablet_copy_service.cc:161] P 91a8ca99f2984301af02199cdb5d78fa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bd329c86b4f84d6b8a67f0b0ac80fbcb at {username='oryx', principal='oryx/127.5.78.60@KRBTEST.COM'} at 127.0.0.1:38802: session id = bd329c86b4f84d6b8a67f0b0ac80fbcb-00000000000000000000000000000000
I20260926 20:46:36.321620 11615 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 20:46:36.323237 12157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 20:46:36.328027 12157 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: tablet copy: Starting download of 0 data blocks...
I20260926 20:46:36.328325 12157 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: tablet copy: Starting download of 1 WAL segments...
I20260926 20:46:36.330543 12157 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 20:46:36.333681 11615 tablet_copy_service.cc:342] P 91a8ca99f2984301af02199cdb5d78fa: Request end of tablet copy session bd329c86b4f84d6b8a67f0b0ac80fbcb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.78.60@KRBTEST.COM'} at 127.0.0.1:38802
I20260926 20:46:36.333818 11615 tablet_copy_service.cc:433] P 91a8ca99f2984301af02199cdb5d78fa: ending tablet copy session bd329c86b4f84d6b8a67f0b0ac80fbcb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bd329c86b4f84d6b8a67f0b0ac80fbcb
I20260926 20:46:36.343717 12157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:36.346362 12359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:36.347626 12360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:36.354664 12362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:36.355055 12157 server_base.cc:1034] running on GCE node
I20260926 20:46:36.355433 12157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:36.356043 12157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:36.357239 12157 hybrid_clock.cc:648] HybridClock initialized: now 1790455596357186 us; error 47 us; skew 500 ppm
I20260926 20:46:36.357471 12157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:36.359373 12157 init.cc:377] Logged in from keytab as oryx/127.5.78.60@KRBTEST.COM (short username oryx)
I20260926 20:46:36.360550 12157 webserver.cc:460] Webserver started at http://127.5.78.60:39301/ using document root <none> and password file <none>
I20260926 20:46:36.360841 12157 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:36.360953 12157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:36.363145 12157 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 20:46:36.364799 12369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:36.365235 12157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260926 20:46:36.365375 12157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
format_stamp: "Formatted at 2026-09-26 20:46:36 on dist-test-slave-z7j8"
server_key: "8c162798692c8ed85dd39bf269f6d737"
server_key_iv: "acf6a505c0fe29271511df2d7b6b9f3b"
server_key_version: "encryptionkey@0"
I20260926 20:46:36.365576 12157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:36.376915 12157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:36.380383 12157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:36.411552 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11656
I20260926 20:46:36.428202 12157 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.60:46291
I20260926 20:46:36.428375 12420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.60:46291 every 8 connection(s)
I20260926 20:46:36.429177 12157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
I20260926 20:46:36.430564 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:32775
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:38333
--webserver_port=45501
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:46:36.435158 12421 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:46:36.439453 12421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Bootstrap starting.
I20260926 20:46:36.459829 12421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:46:36.460340 12421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: Bootstrap complete.
I20260926 20:46:36.461364 12421 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } }
I20260926 20:46:36.461659 12421 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bd329c86b4f84d6b8a67f0b0ac80fbcb, State: Initialized, Role: LEARNER
I20260926 20:46:36.461905 12421 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } }
I20260926 20:46:36.462942 12421 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:36.462692 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.463184 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: This master's current role is: LEARNER
I20260926 20:46:36.470642 12436 catalog_manager.cc:1259] Loaded cluster ID: 8cd0707f941e4faa8950b9baa9b5c7d2
I20260926 20:46:36.470762 12436 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: loading cluster ID for follower catalog manager: success
I20260926 20:46:36.473201 12436 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: acquiring CA information for follower catalog manager: success
I20260926 20:46:36.475191 12436 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:46:36.769073 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.761175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.5.78.60:46291)
0926 20:46:36.761857 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.761864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.761882 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:46:36.761951 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.761957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.762039 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.762157 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:36.762756 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.764335 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.767228 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.768093 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.768203 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.768384 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:36.768695 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 20:46:36.768803 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":390,"threads_started":1}
I20260926 20:46:36.769184 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.760922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33282)
0926 20:46:36.761176 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.761197 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.761327 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:36.762232 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:36.762240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:36.762283 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:36.762585 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.762601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.764526 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.764534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.767012 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:36.767031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:36.768292 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:36.768298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:36.768511 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:36.768553 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:46:36.768667 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 20:46:36.768839 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260926 20:46:36.770256 12396 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 20:46:36.771003 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a8ca99f2984301af02199cdb5d78fa. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:36.771333 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: This master's current role is: LEARNER
W20260926 20:46:36.868655 12423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:36.869016 12423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:36.869207 12423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:36.878958 12423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:36.879133 12423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:36.879335 12423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:36.890208 12423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:32775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=45501
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:36.891575 12423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:36.892922 12423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:36.906698 12442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:36.906677 12443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:36.906770 12445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:36.907428 12423 server_base.cc:1034] running on GCE node
I20260926 20:46:36.907985 12423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:36.909062 12423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:36.910328 12423 hybrid_clock.cc:648] HybridClock initialized: now 1790455596910259 us; error 52 us; skew 500 ppm
I20260926 20:46:36.910720 12423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:36 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:36.914034 12423 init.cc:377] Logged in from keytab as oryx/127.5.78.1@KRBTEST.COM (short username oryx)
I20260926 20:46:36.915833 12423 webserver.cc:460] Webserver started at http://127.5.78.1:45501/ using document root <none> and password file <none>
I20260926 20:46:36.916615 12423 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:36.916805 12423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:36.922819 12423 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:46:36.925714 12452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:36.927425 12423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:36.927647 12423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "e0f2e1e5187f475ba35eeec7369b743c"
format_stamp: "Formatted at 2026-09-26 20:46:33 on dist-test-slave-z7j8"
server_key: "1543a2b2e00ddcb1c242be21d391b3e8"
server_key_iv: "ff1b4c9eb09eec3b2ac821e0f9d56503"
server_key_version: "encryptionkey@0"
I20260926 20:46:36.928812 12423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:36.947158 12423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:36.951680 12423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:36.952008 12423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:36.953007 12423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:36.954478 12423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:36.954608 12423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:36.954749 12423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:36.954838 12423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:36.989662 12423 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:32775
I20260926 20:46:36.989725 12564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:32775 every 8 connection(s)
I20260926 20:46:36.991217 12423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:36.999480 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 12423
I20260926 20:46:36.999882 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11790
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.60@KRBTEST.COM
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455596, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.2@KRBTEST.COM
I20260926 20:46:37.026963 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.994328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:38763 (local address 127.5.78.60:46291)
0926 20:46:36.994550 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:36.994557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:36.995952 (+ 1395us) server_negotiation.cc:407] Connection header received
0926 20:46:36.999270 (+ 3318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:36.999281 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:36.999417 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:36.999501 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.009811 (+ 10310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.010901 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.013172 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.013865 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.013987 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.019359 (+ 5372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.019368 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.019375 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.019423 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.023306 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.024320 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.024327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.024331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.024490 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.025040 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.025045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.025048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.025319 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.025477 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.026337 (+ 860us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.026589 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:46:37.029027 12569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.994308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:55815 (local address 127.5.78.61:45599)
0926 20:46:37.005241 (+ 10933us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.005245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.007374 (+ 2129us) server_negotiation.cc:407] Connection header received
0926 20:46:37.007455 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.007462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.007580 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.007695 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.009618 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.010879 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.013301 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.013989 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.014113 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.020919 (+ 6806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.020942 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.020947 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.020998 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.024498 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.026899 (+ 2401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.026907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.026911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.027079 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.027550 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.027556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.027560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.027897 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.028059 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.028445 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.028677 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10825,"thread_start_us":10723,"threads_started":1}
I20260926 20:46:37.029111 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.994903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.5.78.1:38763)
0926 20:46:36.995795 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:36.995840 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:36.998850 (+ 3010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:37.000386 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:37.000402 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:37.008124 (+ 7722us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:37.009344 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.009379 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.011096 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.011101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.013010 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.013021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.014320 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.014330 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.014761 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.015606 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:37.015642 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:37.019065 (+ 3423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:37.023509 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.023521 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.023543 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.024164 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.024639 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.024643 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.024647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.024930 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.025469 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:37.025477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:37.025963 (+ 486us) client_negotiation.cc:769] Sending connection context
0926 20:46:37.026251 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 20:46:37.026910 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":86,"thread_start_us":384,"threads_started":1}
I20260926 20:46:37.029598 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.994903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.5.78.1:55815)
0926 20:46:37.007014 (+ 12111us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:37.007040 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:37.007206 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:37.007904 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:37.007910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:37.008958 (+ 1048us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:37.009427 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.009447 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.011077 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.011083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.013151 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.013165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.014341 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.014346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.015645 (+ 1299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.016249 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:37.016271 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:37.020661 (+ 4390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:37.025939 (+ 5278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.025944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.025947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.026539 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.027218 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.027223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.027226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.027404 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.028021 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:37.028026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:37.028174 (+ 148us) client_negotiation.cc:769] Sending connection context
0926 20:46:37.028331 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 20:46:37.028616 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11935,"mutex_wait_us":1082,"spinlock_wait_cycles":28416,"thread_start_us":11582,"threads_started":1}
I20260926 20:46:37.030915 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.001319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.5.78.2:40679)
0926 20:46:37.008822 (+ 7503us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.008833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.008871 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:46:37.008963 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.008971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.009266 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.011779 (+ 2513us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.012046 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.013562 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.015396 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.016343 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.020700 (+ 4357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.022549 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.022570 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.022593 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.022644 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.026800 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.027692 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.027701 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.027704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.027857 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.028306 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.028313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.028319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.028723 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.029033 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.030261 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.030547 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7317,"thread_start_us":8184,"threads_started":1}
I20260926 20:46:37.031215 12567 heartbeater.cc:347] Connected to a master server at 127.5.78.61:45599
I20260926 20:46:37.031324 12565 heartbeater.cc:347] Connected to a master server at 127.5.78.60:46291
I20260926 20:46:37.031570 12565 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:37.031574 12567 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:37.031782 12572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.996416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:32929)
0926 20:46:37.007767 (+ 11351us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:37.007786 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:37.007904 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:37.008252 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:37.008259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:37.009009 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:37.009467 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.009481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.011013 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.011018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.014531 (+ 3513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.014545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.015578 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.015585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.015786 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.016762 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:37.016784 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:37.021580 (+ 4796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:37.026681 (+ 5101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.026685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.026688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.027693 (+ 1005us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.029842 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.029847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.029850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.030040 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.030786 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:37.030790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:37.030920 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 20:46:37.031055 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 20:46:37.031340 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11233,"mutex_wait_us":1591,"thread_start_us":11096,"threads_started":1}
I20260926 20:46:37.032657 12567 heartbeater.cc:511] Master 127.5.78.61:45599 requested a full tablet report, sending...
I20260926 20:46:37.032706 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:36.996759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:32929 (local address 127.5.78.62:38333)
0926 20:46:37.007333 (+ 10574us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.007344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.007883 (+ 539us) server_negotiation.cc:407] Connection header received
0926 20:46:37.008010 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.008017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.008103 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.008238 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.009643 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.010838 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.014664 (+ 3826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.015277 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.015421 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.021867 (+ 6446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.021874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.021879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.021925 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.025729 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.029503 (+ 3774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.029509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.029512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.029685 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.030186 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.030192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.030198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.030574 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.030724 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.032105 (+ 1381us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.032339 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10402,"thread_start_us":10322,"threads_started":1}
I20260926 20:46:37.032737 11913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:37.032657 12565 heartbeater.cc:511] Master 127.5.78.60:46291 requested a full tablet report, sending...
I20260926 20:46:37.034200 12566 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38333
I20260926 20:46:37.034335 12566 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:37.034744 12566 heartbeater.cc:511] Master 127.5.78.62:38333 requested a full tablet report, sending...
I20260926 20:46:37.035293 12196 ts_manager.cc:194] Registered new tserver with Master: e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:37.035410 12386 ts_manager.cc:194] Registered new tserver with Master: e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:37.035873 11595 ts_manager.cc:194] Re-registered known tserver with Master: e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:37.036974 11595 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:32929
I20260926 20:46:37.122270 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11790
I20260926 20:46:37.140137 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:40679
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:38333
--webserver_port=46167
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:46:37.193580 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: NON_VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: true } } }
I20260926 20:46:37.194043 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:37.196060 12579 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: attempting to promote NON_VOTER bd329c86b4f84d6b8a67f0b0ac80fbcb to VOTER
I20260926 20:46:37.197005 12579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:37.199352 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Refusing update from remote peer 91a8ca99f2984301af02199cdb5d78fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 20:46:37.199370 12396 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 LEARNER]: Refusing update from remote peer 91a8ca99f2984301af02199cdb5d78fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 20:46:37.200055 12579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 20:46:37.200465 12578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 20:46:37.204787 12344 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.207012 12396 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.208413 12206 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.207979 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.211364 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:37.211705 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.211961 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:37.211725 12579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.212244 12579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:37.212100 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.212343 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:46:37.213384 12582 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a8ca99f2984301af02199cdb5d78fa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:37.213652 12582 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 20:46:37.494657 12577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:37.495015 12577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:37.495187 12577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:37.503055 12577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:37.503213 12577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:37.503405 12577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:37.513628 12577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:40679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=46167
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:37.515024 12577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:37.516353 12577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:37.529707 12588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:37.529850 12587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:37.530296 12590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:37.530709 12577 server_base.cc:1034] running on GCE node
I20260926 20:46:37.531307 12577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:37.532264 12577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:37.533470 12577 hybrid_clock.cc:648] HybridClock initialized: now 1790455597533421 us; error 43 us; skew 500 ppm
I20260926 20:46:37.533756 12577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455597, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:37.536826 12577 init.cc:377] Logged in from keytab as oryx/127.5.78.2@KRBTEST.COM (short username oryx)
I20260926 20:46:37.538200 12577 webserver.cc:460] Webserver started at http://127.5.78.2:46167/ using document root <none> and password file <none>
I20260926 20:46:37.538908 12577 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:37.539018 12577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:37.544276 12577 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 20:46:37.547065 12597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:37.548404 12577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:37.548533 12577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "65dce56cb48f4dbea72fec3593fe18fe"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "4e8d3858f306d0508204b2e2fd56f8d7"
server_key_iv: "f250a46b2e1968d9c5c442cefbdbe044"
server_key_version: "encryptionkey@0"
I20260926 20:46:37.549471 12577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:37.575730 12577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:37.579984 12577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:37.580268 12577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:37.581254 12577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:37.582710 12577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:37.582798 12577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:37.582904 12577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:37.582988 12577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:37.618288 12577 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:40679
I20260926 20:46:37.618372 12709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:40679 every 8 connection(s)
I20260926 20:46:37.619668 12577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:37.623217 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 12577
I20260926 20:46:37.623489 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11924
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455597, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455597, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.60@KRBTEST.COM
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.3@KRBTEST.COM
Sep 26 20:46:37 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455597, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:37.662829 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.622347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:44867 (local address 127.5.78.61:45599)
0926 20:46:37.628891 (+ 6544us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.628901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.635738 (+ 6837us) server_negotiation.cc:407] Connection header received
0926 20:46:37.644911 (+ 9173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.644917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.644998 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.645077 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.646200 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.647251 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.649140 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.649756 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.649846 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.654782 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.654791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.654800 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.654844 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.658733 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.659841 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.659846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.659850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.659999 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.660528 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.660533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.660539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.660926 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.661041 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.662213 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.662468 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6368,"thread_start_us":6509,"threads_started":1}
I20260926 20:46:37.664196 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.623200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.5.78.2:44867)
0926 20:46:37.635517 (+ 12317us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:37.635600 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:37.644712 (+ 9112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:37.645273 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:37.645278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:37.645702 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:37.646027 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.646038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.647417 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.647422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.648966 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.648980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.650149 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.650155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.650283 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.650933 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:37.650954 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:37.654490 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:37.658901 (+ 4411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.658911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.658930 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.659691 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.660142 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.660147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.660150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.660396 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.661147 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:37.661156 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:37.661755 (+ 599us) client_negotiation.cc:769] Sending connection context
0926 20:46:37.663100 (+ 1345us) client_negotiation.cc:240] Negotiation successful
0926 20:46:37.663400 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11866,"mutex_wait_us":67,"spinlock_wait_cycles":15488,"thread_start_us":10518,"threads_started":1}
I20260926 20:46:37.664932 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.622502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:59239 (local address 127.5.78.60:46291)
0926 20:46:37.640436 (+ 17934us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.640445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.640473 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:46:37.642078 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.642087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.642194 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.642308 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.644547 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.645748 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.648357 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.649017 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.649147 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.654884 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.654891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.654898 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.654943 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.658753 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.660015 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.660020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.660025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.660182 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.660588 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.660594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.660598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.660909 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.661065 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.664283 (+ 3218us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.664519 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17741,"thread_start_us":17967,"threads_started":1}
I20260926 20:46:37.665438 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.638858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:33975)
0926 20:46:37.643190 (+ 4332us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:37.643208 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:37.643331 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:37.648187 (+ 4856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:37.648192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:37.648803 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:37.649141 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.649160 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.650575 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.650582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.652448 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.652464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.653693 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.653701 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.653998 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.654621 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:37.654646 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:37.657210 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:37.661431 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.661437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.661444 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.661976 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.662517 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.662522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.662526 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.662698 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.663709 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:37.663714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:37.663872 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 20:46:37.664009 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 20:46:37.664674 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4188,"spinlock_wait_cycles":32000,"thread_start_us":5652,"threads_started":1}
I20260926 20:46:37.665689 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.623751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.5.78.2:59239)
0926 20:46:37.638692 (+ 14941us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:37.638718 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:37.641521 (+ 2803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:37.642805 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:37.642822 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:37.643746 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:37.644366 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.644384 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.645976 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.645982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.648178 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:37.648193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.649448 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.649455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:37.649737 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.650575 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:37.650608 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:37.654697 (+ 4089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:37.658901 (+ 4204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.658911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.658930 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.659866 (+ 936us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.660307 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:37.660311 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:37.660314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:37.660459 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:37.661147 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:37.661156 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:37.661755 (+ 599us) client_negotiation.cc:769] Sending connection context
0926 20:46:37.662107 (+ 352us) client_negotiation.cc:240] Negotiation successful
0926 20:46:37.662689 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14753,"mutex_wait_us":82,"spinlock_wait_cycles":2560,"thread_start_us":11322,"threads_started":1}
I20260926 20:46:37.665889 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.625077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.5.78.3:38237)
0926 20:46:37.640991 (+ 15914us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.641002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.641033 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:46:37.641106 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.641117 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.641474 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.644589 (+ 3115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.644947 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.646979 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.648940 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.649851 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.656115 (+ 6264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.657199 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.657216 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.657234 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.657282 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.661404 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.662310 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.662319 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.662322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.662542 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.663047 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.663052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.663055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.663449 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.664925 (+ 1476us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.665104 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.665408 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15713,"thread_start_us":16592,"threads_started":1}
I20260926 20:46:37.666133 12712 heartbeater.cc:347] Connected to a master server at 127.5.78.61:45599
I20260926 20:46:37.666523 12712 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:37.667163 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:37.635771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:33975 (local address 127.5.78.62:38333)
0926 20:46:37.647595 (+ 11824us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:37.647602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:37.647620 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:46:37.647677 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:37.647682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:37.647751 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:37.647858 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:37.649320 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.650355 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.652659 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:37.653463 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:37.654324 (+ 861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:37.657417 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:37.657436 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:37.657440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:37.657480 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:37.661257 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.662145 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.662165 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.662169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.662324 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:37.662843 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:37.662848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:37.662852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:37.663583 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:37.666284 (+ 2701us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:37.666548 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 20:46:37.666785 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11686,"thread_start_us":12163,"threads_started":1}
I20260926 20:46:37.667361 12047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:37.667423 12712 heartbeater.cc:511] Master 127.5.78.61:45599 requested a full tablet report, sending...
I20260926 20:46:37.667914 12711 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38333
I20260926 20:46:37.668040 12711 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:37.668505 12711 heartbeater.cc:511] Master 127.5.78.62:38333 requested a full tablet report, sending...
I20260926 20:46:37.668736 12710 heartbeater.cc:347] Connected to a master server at 127.5.78.60:46291
I20260926 20:46:37.668883 12710 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:37.669315 12710 heartbeater.cc:511] Master 127.5.78.60:46291 requested a full tablet report, sending...
I20260926 20:46:37.669834 12196 ts_manager.cc:194] Registered new tserver with Master: 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679)
I20260926 20:46:37.670037 11595 ts_manager.cc:194] Re-registered known tserver with Master: 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679)
I20260926 20:46:37.670699 12386 ts_manager.cc:194] Registered new tserver with Master: 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679)
I20260926 20:46:37.671169 11595 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:33975
I20260926 20:46:37.756335 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11924
I20260926 20:46:37.773810 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:38237
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:38333
--webserver_port=45591
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:46:38.040081 12566 heartbeater.cc:503] Master 127.5.78.62:38333 was elected leader, sending a full tablet report...
W20260926 20:46:38.105384 12723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:38.105710 12723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:38.105872 12723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:38.113906 12723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:38.114053 12723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:38.114248 12723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:38.124372 12723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:38237
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=45591
--tserver_master_addrs=127.5.78.62:38333,127.5.78.61:45599,127.5.78.60:46291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:38.126119 12723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:38.127861 12723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:38.141732 12728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:38.141772 12731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:38.142068 12729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:38.142721 12723 server_base.cc:1034] running on GCE node
I20260926 20:46:38.143374 12723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:38.144284 12723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:38.145515 12723 hybrid_clock.cc:648] HybridClock initialized: now 1790455598145453 us; error 57 us; skew 500 ppm
I20260926 20:46:38.145825 12723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:38 dist-test-slave-z7j8 krb5kdc[11550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455598, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:38.148990 12723 init.cc:377] Logged in from keytab as oryx/127.5.78.3@KRBTEST.COM (short username oryx)
I20260926 20:46:38.150561 12723 webserver.cc:460] Webserver started at http://127.5.78.3:45591/ using document root <none> and password file <none>
I20260926 20:46:38.151396 12723 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:38.151525 12723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:38.158303 12723 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:46:38.161773 12738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:38.163647 12723 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:46:38.163825 12723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc"
format_stamp: "Formatted at 2026-09-26 20:46:34 on dist-test-slave-z7j8"
server_key: "cec95cbfc877509972053ccb2917e897"
server_key_iv: "fc1cf18766d8b73841994533ae03fa70"
server_key_version: "encryptionkey@0"
I20260926 20:46:38.164912 12723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:38.180919 12723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:38.185439 12723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:38.185772 12723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:38.186928 12723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:38.188335 12723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:38.188453 12723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:38.188585 12723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:38.188675 12723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:38.222927 12723 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38237
I20260926 20:46:38.223012 12850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38237 every 8 connection(s)
I20260926 20:46:38.224427 12723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:38.234826 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 12723
Sep 26 20:46:38 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455598, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
Sep 26 20:46:38 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455598, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.61@KRBTEST.COM
Sep 26 20:46:38 dist-test-slave-z7j8 krb5kdc[11550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455598, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.60@KRBTEST.COM
I20260926 20:46:38.258963 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.227386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:51375 (local address 127.5.78.61:45599)
0926 20:46:38.227922 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.227926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.229313 (+ 1387us) server_negotiation.cc:407] Connection header received
0926 20:46:38.232240 (+ 2927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.232246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.232323 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.232434 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.236148 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.237246 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.240034 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.240689 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.240820 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.247004 (+ 6184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.247015 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.247023 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.247078 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.252703 (+ 5625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.255769 (+ 3066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.255778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.255784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.255997 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.256719 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.256723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.256726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.257053 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.257172 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.258236 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.258507 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":1005,"threads_started":1}
I20260926 20:46:38.260052 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.227354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:60349 (local address 127.5.78.62:38333)
0926 20:46:38.227805 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.227811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.229218 (+ 1407us) server_negotiation.cc:407] Connection header received
0926 20:46:38.231704 (+ 2486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.231709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.231808 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.231928 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.234566 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.235706 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.238277 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.238929 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.239035 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.244225 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.244235 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.244242 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.244295 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.255520 (+ 11225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.256405 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.256411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.256415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.256558 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.257259 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.257264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.257267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.257642 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.257784 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.259438 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.259686 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":226,"threads_started":1}
I20260926 20:46:38.260735 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.228024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.5.78.3:51375)
0926 20:46:38.229158 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.229175 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.231414 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.232484 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.232490 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.233372 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:38.234373 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.234423 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.237391 (+ 2968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.237396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.239869 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.239883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.240906 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.240912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.241088 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.242090 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:38.242112 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:38.246017 (+ 3905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:38.252934 (+ 6917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:38.252949 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:38.252977 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:38.253873 (+ 896us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:38.256112 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:38.256119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:38.256125 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:38.256403 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:38.257188 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:38.257198 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:38.257771 (+ 573us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.258102 (+ 331us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.258842 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":141,"spinlock_wait_cycles":124672,"thread_start_us":656,"threads_started":1}
I20260926 20:46:38.261600 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.228024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:60349)
0926 20:46:38.229044 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.229092 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.231414 (+ 2322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.232185 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.232225 (+ 40us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.233372 (+ 1147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:38.234366 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.234408 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.235884 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.235891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.238114 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.238133 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.239171 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.239181 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.239569 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.240486 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:38.240508 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:38.243742 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:38.255686 (+ 11944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:38.255694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:38.255700 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:38.256254 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:38.256670 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:38.256677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:38.256683 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:38.256867 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:38.257749 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:38.257752 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:38.257869 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.258102 (+ 233us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.261286 (+ 3184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":2996,"spinlock_wait_cycles":29568,"thread_start_us":433,"threads_started":1}
I20260926 20:46:38.263088 12853 heartbeater.cc:347] Connected to a master server at 127.5.78.61:45599
I20260926 20:46:38.263458 12853 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:38.264415 12853 heartbeater.cc:511] Master 127.5.78.61:45599 requested a full tablet report, sending...
I20260926 20:46:38.265265 12852 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38333
I20260926 20:46:38.265403 12852 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:38.265854 12852 heartbeater.cc:511] Master 127.5.78.62:38333 requested a full tablet report, sending...
I20260926 20:46:38.266288 12196 ts_manager.cc:194] Registered new tserver with Master: 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237)
I20260926 20:46:38.267849 11595 ts_manager.cc:194] Re-registered known tserver with Master: 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237)
I20260926 20:46:38.269182 11595 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:60349
I20260926 20:46:38.272014 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.229866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:55279 (local address 127.5.78.60:46291)
0926 20:46:38.230737 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.230742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.230757 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:46:38.231704 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.231710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.231793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.231976 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.234632 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.235714 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.238216 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.238871 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.239007 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.257336 (+ 18329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.257342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.257348 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.257391 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.263943 (+ 6552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.265674 (+ 1731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.265680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.265683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.265878 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.266447 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.266455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.266461 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.266925 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.267360 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.271214 (+ 3854us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.271700 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":675,"threads_started":1}
I20260926 20:46:38.272413 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.229747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.5.78.3:55279)
0926 20:46:38.230331 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.230346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.231430 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.232185 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.232215 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.233465 (+ 1250us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:38.234458 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.234469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.235887 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.235895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.238034 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.238048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.239252 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.239264 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.239562 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.249963 (+ 10401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:38.249994 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:38.255512 (+ 5518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:38.264355 (+ 8843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:38.264362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:38.264369 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:38.265475 (+ 1106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:38.266051 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:38.266056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:38.266059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:38.266244 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:38.269817 (+ 3573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:38.269823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:38.269989 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.270156 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.270495 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":307,"spinlock_wait_cycles":28416,"thread_start_us":312,"threads_started":1}
I20260926 20:46:38.273339 12851 heartbeater.cc:347] Connected to a master server at 127.5.78.60:46291
I20260926 20:46:38.273514 12851 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:38.273886 12851 heartbeater.cc:511] Master 127.5.78.60:46291 requested a full tablet report, sending...
I20260926 20:46:38.275017 12386 ts_manager.cc:194] Registered new tserver with Master: 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237)
I20260926 20:46:38.283699 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.244006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.5.78.60:46291)
0926 20:46:38.256405 (+ 12399us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.256411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.256429 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:46:38.256486 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.256494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.256570 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.256691 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.261585 (+ 4894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.265930 (+ 4345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.269357 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.269968 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.270089 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.274589 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.274599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.274607 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.274660 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.280525 (+ 5865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.281261 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.281273 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.281275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.281395 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.281790 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.281795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.281798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.282205 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.282464 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.283186 (+ 722us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.283352 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12240,"thread_start_us":12840,"threads_started":1}
I20260926 20:46:38.285949 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.242169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.5.78.62:38333)
0926 20:46:38.260537 (+ 18368us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.260544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.260562 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:46:38.260606 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.260610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.260699 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.260796 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.265003 (+ 4207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.267442 (+ 2439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.269275 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.269957 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.270120 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.278088 (+ 7968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.278097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.278124 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.278176 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.281741 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.282612 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.282618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.282621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.282804 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.284073 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.284101 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.284106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.284497 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.284752 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.285413 (+ 661us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.285664 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18241,"thread_start_us":18747,"threads_started":1}
I20260926 20:46:38.288921 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.246760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.5.78.61:45599)
0926 20:46:38.262895 (+ 16135us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.262902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.271208 (+ 8306us) server_negotiation.cc:407] Connection header received
0926 20:46:38.271784 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.271790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.271866 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.271942 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.275303 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.276465 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.278511 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.279202 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.279342 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.282823 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.282830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.282838 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.282886 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.285992 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.286794 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.286799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.286802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.286933 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.287384 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.287388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.287390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.287662 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.287951 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.288292 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.288530 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16025,"thread_start_us":28358,"threads_started":1}
I20260926 20:46:38.290607 11595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:38.297583 11595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:38.335783 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.318994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.5.78.2:40679)
0926 20:46:38.326834 (+ 7840us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.326845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.326876 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:46:38.326981 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.326991 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.327238 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.327515 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:38.328287 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.329728 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.332735 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.334210 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.334346 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.335190 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.335258 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.335418 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7684,"thread_start_us":8045,"threads_started":1}
I20260926 20:46:38.336189 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.318445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.5.78.3:38237)
0926 20:46:38.325396 (+ 6951us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.325407 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.326111 (+ 704us) server_negotiation.cc:407] Connection header received
0926 20:46:38.326233 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.326240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.326571 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.326796 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:38.327429 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.329085 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.332129 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.333568 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.333668 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.333823 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.335017 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.335197 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6702,"thread_start_us":6470,"threads_started":1}
I20260926 20:46:38.336818 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.318470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:40679 (local address 127.0.0.1:50996)
0926 20:46:38.326546 (+ 8076us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.326570 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.326746 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.327747 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.327752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.327775 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:38.328102 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.328113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.329887 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.329895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.332549 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.332580 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.334451 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.334458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.334687 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.334722 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.334804 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.336485 (+ 1681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7956,"mutex_wait_us":27,"thread_start_us":7387,"threads_started":1}
I20260926 20:46:38.336905 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.317964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38237 (local address 127.0.0.1:54920)
0926 20:46:38.325770 (+ 7806us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.325798 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.326009 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.326875 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.326879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.326903 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:38.327174 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.327191 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.329234 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.329243 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.331948 (+ 2705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.331963 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.334602 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.334611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.334813 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.334858 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.334947 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.336506 (+ 1559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7616,"mutex_wait_us":1496,"thread_start_us":8337,"threads_started":1}
I20260926 20:46:38.339568 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.327504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:32775 (local address 127.0.0.1:43900)
0926 20:46:38.329681 (+ 2177us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.329698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.329812 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.332115 (+ 2303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.332120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.332143 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:38.332688 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.332700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.334525 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.334530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.337138 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.337151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.338712 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.338719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.338902 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.338987 (+ 85us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.339083 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.339220 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2038,"mutex_wait_us":398,"thread_start_us":2768,"threads_started":1}
I20260926 20:46:38.339579 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.326954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.5.78.1:32775)
0926 20:46:38.331196 (+ 4242us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.331207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.331238 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:46:38.331332 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.331340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.331668 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.331867 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:38.332859 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.334339 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.337295 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.338485 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.338619 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.338820 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.339141 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.339307 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4083,"thread_start_us":3962,"threads_started":1}
I20260926 20:46:38.340811 12645 tablet_service.cc:1467] Processing CreateTablet for tablet c3c3229e2ec54a7da602d0ed707a7bb2 (DEFAULT_TABLE table=test-table [id=e6cd99fc24d446bfb0115863d9784aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:38.340777 12786 tablet_service.cc:1467] Processing CreateTablet for tablet c3c3229e2ec54a7da602d0ed707a7bb2 (DEFAULT_TABLE table=test-table [id=e6cd99fc24d446bfb0115863d9784aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:38.342095 12645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3c3229e2ec54a7da602d0ed707a7bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:38.342200 12786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3c3229e2ec54a7da602d0ed707a7bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:38.343881 12500 tablet_service.cc:1467] Processing CreateTablet for tablet c3c3229e2ec54a7da602d0ed707a7bb2 (DEFAULT_TABLE table=test-table [id=e6cd99fc24d446bfb0115863d9784aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:38.345331 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3c3229e2ec54a7da602d0ed707a7bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:38.355963 12887 tablet_bootstrap.cc:492] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Bootstrap starting.
I20260926 20:46:38.357470 12888 tablet_bootstrap.cc:492] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Bootstrap starting.
I20260926 20:46:38.359865 12889 tablet_bootstrap.cc:492] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Bootstrap starting.
I20260926 20:46:38.360247 12887 tablet_bootstrap.cc:654] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:38.360855 12888 tablet_bootstrap.cc:654] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:38.361621 12887 log.cc:824] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:38.362038 12888 log.cc:824] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:38.363045 12889 tablet_bootstrap.cc:654] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:38.364223 12889 log.cc:824] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:38.369360 12887 tablet_bootstrap.cc:492] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: No bootstrap required, opened a new log
I20260926 20:46:38.369416 12889 tablet_bootstrap.cc:492] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: No bootstrap required, opened a new log
I20260926 20:46:38.369416 12888 tablet_bootstrap.cc:492] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: No bootstrap required, opened a new log
I20260926 20:46:38.369660 12887 ts_tablet_manager.cc:1397] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260926 20:46:38.369716 12889 ts_tablet_manager.cc:1397] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260926 20:46:38.369699 12888 ts_tablet_manager.cc:1397] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260926 20:46:38.377007 12887 raft_consensus.cc:348] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.377403 12887 raft_consensus.cc:374] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:38.377494 12887 raft_consensus.cc:729] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dce56cb48f4dbea72fec3593fe18fe, State: Initialized, Role: FOLLOWER
I20260926 20:46:38.378124 12887 consensus_queue.cc:260] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.379958 12887 ts_tablet_manager.cc:1428] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260926 20:46:38.380203 12711 heartbeater.cc:503] Master 127.5.78.62:38333 was elected leader, sending a full tablet report...
I20260926 20:46:38.380455 12889 raft_consensus.cc:348] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.380455 12888 raft_consensus.cc:348] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.380893 12888 raft_consensus.cc:374] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:38.380892 12889 raft_consensus.cc:374] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:38.380986 12889 raft_consensus.cc:729] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f2e1e5187f475ba35eeec7369b743c, State: Initialized, Role: FOLLOWER
I20260926 20:46:38.380986 12888 raft_consensus.cc:729] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb3ffe654b54778b4d0b4eb7d9ecbbc, State: Initialized, Role: FOLLOWER
I20260926 20:46:38.381601 12889 consensus_queue.cc:260] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.381601 12888 consensus_queue.cc:260] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.386058 12852 heartbeater.cc:503] Master 127.5.78.62:38333 was elected leader, sending a full tablet report...
I20260926 20:46:38.388070 12888 ts_tablet_manager.cc:1428] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Time spent starting tablet: real 0.018s user 0.008s sys 0.011s
I20260926 20:46:38.388736 12889 ts_tablet_manager.cc:1428] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Time spent starting tablet: real 0.019s user 0.008s sys 0.009s
W20260926 20:46:38.393878 12713 tablet.cc:2369] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:46:38.476788 12854 tablet.cc:2369] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:46:38.494915 12568 tablet.cc:2369] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:46:38.573833 12893 raft_consensus.cc:482] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:38.574136 12893 raft_consensus.cc:504] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.575825 12893 leader_election.cc:288] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237), e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:38.583204 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.576325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:52189 (local address 127.5.78.3:38237)
0926 20:46:38.576593 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.576602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.576984 (+ 382us) server_negotiation.cc:407] Connection header received
0926 20:46:38.577212 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.577216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.577292 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.577383 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:38.578095 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.579175 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.581209 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.582179 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.582286 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.582372 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.582814 (+ 442us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.582914 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:46:38.583381 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.576252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38237 (local address 127.5.78.2:52189)
0926 20:46:38.576867 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.576885 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.577069 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.577478 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.577487 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.577520 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:38.577937 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.577951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.579304 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.579310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.581037 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.581049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.582433 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.582440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.582623 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.582669 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.582790 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.582932 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":24,"thread_start_us":313,"threads_started":1}
I20260926 20:46:38.583933 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.577117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:53193 (local address 127.5.78.1:32775)
0926 20:46:38.577389 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.577395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.577813 (+ 418us) server_negotiation.cc:407] Connection header received
0926 20:46:38.578017 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.578034 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.578118 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.578215 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:38.578752 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.579833 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.581822 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.582912 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.583036 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.583141 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.583533 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.583680 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 20:46:38.584512 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.576946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:32775 (local address 127.5.78.2:53193)
0926 20:46:38.577681 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:38.577696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:38.577871 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:38.578254 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:38.578259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:38.578284 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:38.578571 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.578579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.579982 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.579986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.581634 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:38.581648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.583138 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.583144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:38.583335 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.583377 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:46:38.584139 (+ 762us) client_negotiation.cc:240] Negotiation successful
0926 20:46:38.584254 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":24,"thread_start_us":432,"threads_started":1}
I20260926 20:46:38.584478 12806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c3229e2ec54a7da602d0ed707a7bb2" candidate_uuid: "65dce56cb48f4dbea72fec3593fe18fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" is_pre_election: true
I20260926 20:46:38.585121 12806 raft_consensus.cc:2454] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dce56cb48f4dbea72fec3593fe18fe in term 0.
I20260926 20:46:38.585595 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c3229e2ec54a7da602d0ed707a7bb2" candidate_uuid: "65dce56cb48f4dbea72fec3593fe18fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f2e1e5187f475ba35eeec7369b743c" is_pre_election: true
I20260926 20:46:38.585817 12601 leader_election.cc:302] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dce56cb48f4dbea72fec3593fe18fe, 8cb3ffe654b54778b4d0b4eb7d9ecbbc; no voters:
I20260926 20:46:38.586115 12520 raft_consensus.cc:2454] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dce56cb48f4dbea72fec3593fe18fe in term 0.
I20260926 20:46:38.586292 12893 raft_consensus.cc:2781] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:38.586470 12893 raft_consensus.cc:482] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:38.586603 12893 raft_consensus.cc:3037] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:38.588790 12893 raft_consensus.cc:504] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.589411 12893 leader_election.cc:288] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [CANDIDATE]: Term 1 election: Requested vote from peers 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237), e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:38.589972 12806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c3229e2ec54a7da602d0ed707a7bb2" candidate_uuid: "65dce56cb48f4dbea72fec3593fe18fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc"
I20260926 20:46:38.590018 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3c3229e2ec54a7da602d0ed707a7bb2" candidate_uuid: "65dce56cb48f4dbea72fec3593fe18fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f2e1e5187f475ba35eeec7369b743c"
I20260926 20:46:38.590184 12806 raft_consensus.cc:3037] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:38.590247 12520 raft_consensus.cc:3037] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:38.592545 12806 raft_consensus.cc:2454] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dce56cb48f4dbea72fec3593fe18fe in term 1.
I20260926 20:46:38.592612 12520 raft_consensus.cc:2454] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dce56cb48f4dbea72fec3593fe18fe in term 1.
I20260926 20:46:38.593029 12601 leader_election.cc:302] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dce56cb48f4dbea72fec3593fe18fe, 8cb3ffe654b54778b4d0b4eb7d9ecbbc; no voters:
I20260926 20:46:38.593375 12893 raft_consensus.cc:2781] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:38.593842 12893 raft_consensus.cc:686] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 LEADER]: Becoming Leader. State: Replica: 65dce56cb48f4dbea72fec3593fe18fe, State: Running, Role: LEADER
I20260926 20:46:38.594458 12893 consensus_queue.cc:237] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } }
I20260926 20:46:38.599980 11593 catalog_manager.cc:5495] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2). New cstate: current_term: 1 leader_uuid: "65dce56cb48f4dbea72fec3593fe18fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } health_report { overall_health: UNKNOWN } } }
I20260926 20:46:38.648388 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.642857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.5.78.2:40679)
0926 20:46:38.643094 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.643098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.643118 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:46:38.643307 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.643311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.643386 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.643512 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:38.643893 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.644590 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.646032 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.646541 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.646647 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.647132 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:38.647690 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:38.647817 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.648000 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.648120 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:46:38.656832 12520 raft_consensus.cc:1261] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Refusing update from remote peer 65dce56cb48f4dbea72fec3593fe18fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:38.656838 12806 raft_consensus.cc:1261] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Refusing update from remote peer 65dce56cb48f4dbea72fec3593fe18fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:38.657820 12898 consensus_queue.cc:1035] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:38.658185 12893 consensus_queue.cc:1035] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:38.688925 11593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:38.689231 11593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:38.695587 11593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91a8ca99f2984301af02199cdb5d78fa: Sending DeleteTablet for 3 replicas of tablet c3c3229e2ec54a7da602d0ed707a7bb2
I20260926 20:46:38.696911 12645 tablet_service.cc:1514] Processing DeleteTablet for tablet c3c3229e2ec54a7da602d0ed707a7bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:38 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:50996
I20260926 20:46:38.697526 12786 tablet_service.cc:1514] Processing DeleteTablet for tablet c3c3229e2ec54a7da602d0ed707a7bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:38 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:54920
I20260926 20:46:38.697546 12500 tablet_service.cc:1514] Processing DeleteTablet for tablet c3c3229e2ec54a7da602d0ed707a7bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:38 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:43900
I20260926 20:46:38.708899 12912 tablet_replica.cc:330] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: stopping tablet replica
I20260926 20:46:38.709354 12912 raft_consensus.cc:2229] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:38.709841 12912 raft_consensus.cc:2258] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:38.711616 12913 tablet_replica.cc:330] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: stopping tablet replica
I20260926 20:46:38.713084 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 11563
I20260926 20:46:38.713714 12913 raft_consensus.cc:2229] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:38.714391 12913 raft_consensus.cc:2258] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:38.714556 11625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:38.717411 12911 tablet_replica.cc:330] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: stopping tablet replica
I20260926 20:46:38.717880 12911 raft_consensus.cc:2229] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:38.718770 12912 ts_tablet_manager.cc:1905] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:38.719022 12913 ts_tablet_manager.cc:1905] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:38.722215 12911 raft_consensus.cc:2258] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:38.726311 12912 ts_tablet_manager.cc:1918] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:38.726548 12912 log.cc:1196] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/c3c3229e2ec54a7da602d0ed707a7bb2
I20260926 20:46:38.727088 12912 ts_tablet_manager.cc:1939] T c3c3229e2ec54a7da602d0ed707a7bb2 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Deleting consensus metadata
I20260926 20:46:38.727699 12911 ts_tablet_manager.cc:1905] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:38.734661 12911 ts_tablet_manager.cc:1918] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:38.734673 12913 ts_tablet_manager.cc:1918] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:38.734879 12911 log.cc:1196] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/c3c3229e2ec54a7da602d0ed707a7bb2
I20260926 20:46:38.734898 12913 log.cc:1196] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/c3c3229e2ec54a7da602d0ed707a7bb2
I20260926 20:46:38.735512 12911 ts_tablet_manager.cc:1939] T c3c3229e2ec54a7da602d0ed707a7bb2 P 65dce56cb48f4dbea72fec3593fe18fe: Deleting consensus metadata
I20260926 20:46:38.735532 12913 ts_tablet_manager.cc:1939] T c3c3229e2ec54a7da602d0ed707a7bb2 P e0f2e1e5187f475ba35eeec7369b743c: Deleting consensus metadata
I20260926 20:46:38.738606 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.702161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.5.78.61:45599)
0926 20:46:38.702363 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.702372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.711115 (+ 8743us) server_negotiation.cc:407] Connection header received
0926 20:46:38.712103 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.712109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.712198 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.712277 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.716989 (+ 4712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.718134 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.720370 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.721081 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.721192 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.730682 (+ 9490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.730692 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.730697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.730756 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.734581 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.735714 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.735722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.735725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.735889 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.736385 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.736391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.736395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.736799 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.737022 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.737968 (+ 946us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.738190 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 20:46:38.752242 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.711388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.5.78.60:46291)
0926 20:46:38.711632 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.711641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.738807 (+ 27166us) server_negotiation.cc:407] Connection header received
0926 20:46:38.739085 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.739091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.739174 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.739264 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.740915 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.741962 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.743546 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.743971 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.744116 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.746848 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.746854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.746858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.746892 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.749512 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.750203 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.750214 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.750216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.750341 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.750814 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.750819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.750824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.751093 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.751394 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.751667 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.751883 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:46:38.890301 11579 catalog_manager.cc:4841] TS 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679): tablet c3c3229e2ec54a7da602d0ed707a7bb2 (table test-table [id=e6cd99fc24d446bfb0115863d9784aa3]) successfully deleted
I20260926 20:46:38.890767 11582 catalog_manager.cc:4841] TS e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775): tablet c3c3229e2ec54a7da602d0ed707a7bb2 (table test-table [id=e6cd99fc24d446bfb0115863d9784aa3]) successfully deleted
I20260926 20:46:38.891058 11582 catalog_manager.cc:4841] TS 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237): tablet c3c3229e2ec54a7da602d0ed707a7bb2 (table test-table [id=e6cd99fc24d446bfb0115863d9784aa3]) successfully deleted
I20260926 20:46:38.891268 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 11563
I20260926 20:46:38.945580 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.926366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.5.78.61:45599)
0926 20:46:38.926593 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.926599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.927925 (+ 1326us) server_negotiation.cc:407] Connection header received
0926 20:46:38.928189 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.928208 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.928294 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.928370 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.930668 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.931823 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.933891 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.934595 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.934698 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.939020 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.939026 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.939031 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.939074 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.942165 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.943249 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.943258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.943264 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.943439 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.943804 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.943812 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.943817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.944233 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.944505 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.944909 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.945182 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 20:46:38.952371 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:38.927331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.5.78.60:46291)
0926 20:46:38.927501 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:38.927507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:38.936064 (+ 8557us) server_negotiation.cc:407] Connection header received
0926 20:46:38.936992 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:38.936998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:38.937079 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:38.937194 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:38.938915 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.940073 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.941931 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:38.942603 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:38.942753 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:38.946060 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:38.946069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:38.946079 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:38.946111 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:38.949555 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.950224 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.950231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.950235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.950374 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:38.950763 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:38.950769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:38.950772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:38.951122 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:38.951358 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:38.951783 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 20:46:38.952001 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:46:39.677035 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:39.676198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:37589)
0926 20:46:39.676716 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:39.676857 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":307,"threads_started":1}
W20260926 20:46:39.677439 12711 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:38333 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:39.683957 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:39.682834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:40203)
0926 20:46:39.683420 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:39.683567 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":314,"threads_started":1}
W20260926 20:46:39.684459 12852 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:38333 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:39.685238 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:39.684569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:56739)
0926 20:46:39.684992 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:39.685106 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":240,"threads_started":1}
W20260926 20:46:39.685652 12566 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:38333 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.196945 12926 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91a8ca99f2984301af02199cdb5d78fa)
I20260926 20:46:40.197041 12927 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91a8ca99f2984301af02199cdb5d78fa)
I20260926 20:46:40.197252 12926 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:40.197372 12927 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:40.198796 12926 leader_election.cc:288] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333), bebec0aad9d04aed8d8d8f7d6ed48653 (127.5.78.61:45599)
I20260926 20:46:40.198807 12927 leader_election.cc:288] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333), bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60:46291)
I20260926 20:46:40.200105 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.198949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39252)
0926 20:46:40.199572 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.199753 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":269,"threads_started":1}
W20260926 20:46:40.201555 12181 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.201736 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.199079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39250)
0926 20:46:40.201156 (+ 2077us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.201457 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1937,"threads_started":1}
W20260926 20:46:40.203186 12371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.203895 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.203326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39258)
0926 20:46:40.203527 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.203683 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 20:46:40.204227 12181 leader_election.cc:334] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.207345 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.206723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39260)
0926 20:46:40.206933 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.207108 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 20:46:40.207612 12371 leader_election.cc:334] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.210074 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.199629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46291 (local address 127.0.0.1:33328)
0926 20:46:40.200394 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.200416 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:40.200582 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:40.203950 (+ 3368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:40.203954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:40.203990 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:40.204448 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:40.204466 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.205756 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.205763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:40.207897 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:40.207911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.209183 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.209189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:40.209385 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:40.209412 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 20:46:40.209531 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 20:46:40.209690 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":58,"thread_start_us":1583,"threads_started":1}
I20260926 20:46:40.210074 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.202447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.5.78.60:46291)
0926 20:46:40.203577 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:40.203583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:40.203602 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:46:40.203673 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:40.203679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:40.203765 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:40.204244 (+ 479us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:40.204822 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.205621 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.208093 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.208999 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.209144 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:40.209247 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:40.209569 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 20:46:40.209703 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":499,"threads_started":1}
I20260926 20:46:40.210506 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.199281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:45599 (local address 127.0.0.1:54062)
0926 20:46:40.202003 (+ 2722us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.202033 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:40.202260 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:40.203453 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:40.203458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:40.203490 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:40.203792 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:40.203812 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.206185 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.206192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:40.208199 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:40.208240 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.209531 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.209538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:40.209737 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:40.209763 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 20:46:40.209912 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 20:46:40.210077 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2581,"thread_start_us":1575,"threads_started":1}
I20260926 20:46:40.210958 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.199544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.5.78.61:45599)
0926 20:46:40.201951 (+ 2407us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:40.201959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:40.202143 (+ 184us) server_negotiation.cc:407] Connection header received
0926 20:46:40.202970 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:40.202978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:40.203064 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:40.203153 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:40.204062 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.206033 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.208408 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:40.209315 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:40.209493 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:40.209621 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:40.209936 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 20:46:40.210093 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":2106,"thread_start_us":1944,"threads_started":1}
I20260926 20:46:40.210991 12396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" is_pre_election: true
I20260926 20:46:40.211412 12396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bebec0aad9d04aed8d8d8f7d6ed48653 in term 1.
I20260926 20:46:40.211633 12206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" is_pre_election: true
I20260926 20:46:40.212049 12181 leader_election.cc:302] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd329c86b4f84d6b8a67f0b0ac80fbcb, bebec0aad9d04aed8d8d8f7d6ed48653; no voters: 91a8ca99f2984301af02199cdb5d78fa
I20260926 20:46:40.212076 12206 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd329c86b4f84d6b8a67f0b0ac80fbcb in term 1.
I20260926 20:46:40.212456 12927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 20:46:40.212596 12927 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91a8ca99f2984301af02199cdb5d78fa)
I20260926 20:46:40.212718 12927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:46:40.212648 12373 leader_election.cc:302] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd329c86b4f84d6b8a67f0b0ac80fbcb, bebec0aad9d04aed8d8d8f7d6ed48653; no voters: 91a8ca99f2984301af02199cdb5d78fa
I20260926 20:46:40.213073 12926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 20:46:40.213231 12926 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91a8ca99f2984301af02199cdb5d78fa)
I20260926 20:46:40.213361 12926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:46:40.215282 12927 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:40.215739 12926 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:40.216027 12927 leader_election.cc:288] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 2 election: Requested vote from peers 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333), bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60:46291)
I20260926 20:46:40.216447 12926 leader_election.cc:288] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [CANDIDATE]: Term 2 election: Requested vote from peers 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333), bebec0aad9d04aed8d8d8f7d6ed48653 (127.5.78.61:45599)
I20260926 20:46:40.217021 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.216539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39284)
0926 20:46:40.216697 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.216769 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 20:46:40.216944 12206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653"
I20260926 20:46:40.216965 12396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
I20260926 20:46:40.217089 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.216179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39272)
0926 20:46:40.216376 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.216896 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 20:46:40.217247 12396 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bebec0aad9d04aed8d8d8f7d6ed48653 in current term 2: Already voted for candidate bd329c86b4f84d6b8a67f0b0ac80fbcb in this term.
I20260926 20:46:40.217260 12206 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bd329c86b4f84d6b8a67f0b0ac80fbcb in current term 2: Already voted for candidate bebec0aad9d04aed8d8d8f7d6ed48653 in this term.
I20260926 20:46:40.218115 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.217731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39286)
0926 20:46:40.217893 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.217974 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 20:46:40.218276 12181 leader_election.cc:334] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.218472 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.218042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39296)
0926 20:46:40.218219 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.218297 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 20:46:40.218480 12181 leader_election.cc:302] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bebec0aad9d04aed8d8d8f7d6ed48653; no voters: 91a8ca99f2984301af02199cdb5d78fa, bd329c86b4f84d6b8a67f0b0ac80fbcb
W20260926 20:46:40.218665 12371 leader_election.cc:334] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:40.218768 12927 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 20:46:40.218791 12371 leader_election.cc:302] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd329c86b4f84d6b8a67f0b0ac80fbcb; no voters: 91a8ca99f2984301af02199cdb5d78fa, bebec0aad9d04aed8d8d8f7d6ed48653
I20260926 20:46:40.219086 12926 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 20:46:40.679190 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.678241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:41587)
0926 20:46:40.678913 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.679018 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":469,"threads_started":1}
I20260926 20:46:40.685895 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.685069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:37303)
0926 20:46:40.685532 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.685677 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":348,"threads_started":1}
I20260926 20:46:40.687157 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:40.686336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:54625)
0926 20:46:40.686912 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:40.687000 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":361,"threads_started":1}
I20260926 20:46:41.681072 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:41.680227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:35839)
0926 20:46:41.680745 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:41.680872 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":239,"threads_started":1}
I20260926 20:46:41.688061 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:41.686938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:58973)
0926 20:46:41.687468 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:41.687830 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":299,"threads_started":1}
I20260926 20:46:41.688650 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:41.687928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:52029)
0926 20:46:41.688343 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:41.688452 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":327,"threads_started":1}
I20260926 20:46:42.396041 12942 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:42.396261 12942 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:42.396862 12942 leader_election.cc:288] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333), bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60:46291)
I20260926 20:46:42.398032 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.397120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39588)
0926 20:46:42.397692 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.397828 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":285,"threads_started":1}
I20260926 20:46:42.397984 12396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" is_pre_election: true
I20260926 20:46:42.398260 12396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bebec0aad9d04aed8d8d8f7d6ed48653 in term 2.
I20260926 20:46:42.398736 12181 leader_election.cc:302] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd329c86b4f84d6b8a67f0b0ac80fbcb, bebec0aad9d04aed8d8d8f7d6ed48653; no voters:
I20260926 20:46:42.399024 12942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 20:46:42.399165 12942 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:42.399271 12942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 2 FOLLOWER]: Advancing to term 3
I20260926 20:46:42.399621 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.399205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39594)
0926 20:46:42.399396 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.399482 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260926 20:46:42.399816 12181 leader_election.cc:334] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:42.401568 12942 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:42.402189 12942 leader_election.cc:288] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 3 election: Requested vote from peers 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333), bd329c86b4f84d6b8a67f0b0ac80fbcb (127.5.78.60:46291)
I20260926 20:46:42.402809 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.402277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39608)
0926 20:46:42.402537 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.402642 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:46:42.403020 12396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb"
I20260926 20:46:42.403288 12396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 2 FOLLOWER]: Advancing to term 3
I20260926 20:46:42.403789 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.403412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39616)
0926 20:46:42.403597 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.403671 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260926 20:46:42.403957 12181 leader_election.cc:334] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
I20260926 20:46:42.405532 12396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bebec0aad9d04aed8d8d8f7d6ed48653 in term 3.
I20260926 20:46:42.406131 12181 leader_election.cc:302] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd329c86b4f84d6b8a67f0b0ac80fbcb, bebec0aad9d04aed8d8d8f7d6ed48653; no voters: 91a8ca99f2984301af02199cdb5d78fa
I20260926 20:46:42.406461 12942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 20:46:42.406841 12942 raft_consensus.cc:686] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [term 3 LEADER]: Becoming Leader. State: Replica: bebec0aad9d04aed8d8d8f7d6ed48653, State: Running, Role: LEADER
I20260926 20:46:42.407208 12942 consensus_queue.cc:237] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } }
I20260926 20:46:42.408939 12945 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebec0aad9d04aed8d8d8f7d6ed48653. Latest consensus state: current_term: 3 leader_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:42.409353 12945 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:42.410043 12947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:42.413666 12947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:42.414582 12947 catalog_manager.cc:1259] Loaded cluster ID: 8cd0707f941e4faa8950b9baa9b5c7d2
I20260926 20:46:42.414688 12947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:42.415817 12947 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:42.416596 12947 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Loaded TSK: 0
I20260926 20:46:42.417408 12947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:46:42.480686 12196 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:42.487562 12196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:42.491927 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.491509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39646)
0926 20:46:42.491666 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.491775 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 20:46:42.492413 12396 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [term 3 FOLLOWER]: Refusing update from remote peer bebec0aad9d04aed8d8d8f7d6ed48653: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260926 20:46:42.493120 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.492675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39660)
0926 20:46:42.492906 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.492995 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 20:46:42.493131 12945 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 20:46:42.493466 12181 consensus_peers.cc:487] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 -> Peer 91a8ca99f2984301af02199cdb5d78fa (127.5.78.62:38333): Couldn't send request to peer 91a8ca99f2984301af02199cdb5d78fa. Status: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 20:46:42.496429 12949 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebec0aad9d04aed8d8d8f7d6ed48653. Latest consensus state: current_term: 3 leader_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:42.496770 12949 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:46:42.497980 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:42.498461 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653 [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:42.499342 12949 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a8ca99f2984301af02199cdb5d78fa" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38333 } } peers { permanent_uuid: "bebec0aad9d04aed8d8d8f7d6ed48653" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 45599 } attrs { promote: false } } peers { permanent_uuid: "bd329c86b4f84d6b8a67f0b0ac80fbcb" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46291 } attrs { promote: false } } }
I20260926 20:46:42.499588 12949 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd329c86b4f84d6b8a67f0b0ac80fbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:46:42.515929 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.508635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.5.78.3:38237)
0926 20:46:42.509070 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.509077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.510013 (+ 936us) server_negotiation.cc:407] Connection header received
0926 20:46:42.510075 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.510078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.510141 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.510249 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:42.510968 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.511756 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.514018 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.514980 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.515130 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.515241 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.515487 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.515624 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":574,"threads_started":1}
I20260926 20:46:42.515971 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.508548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38237 (local address 127.0.0.1:54922)
0926 20:46:42.509782 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.509801 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:42.509928 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:42.510260 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:42.510265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:42.510286 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:42.510758 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.510771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.511942 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.511947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.513865 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.513877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.515147 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.515153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.515328 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.515352 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 20:46:42.515459 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:46:42.515598 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":870,"threads_started":1}
I20260926 20:46:42.516945 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.506901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:40679 (local address 127.0.0.1:51010)
0926 20:46:42.507099 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.507121 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:42.507237 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:42.508150 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:42.508156 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:42.508181 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:42.508513 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.508544 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.510360 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.510370 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.513398 (+ 3028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.513418 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.515923 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.515933 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.516197 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.516264 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 20:46:42.516343 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 20:46:42.516557 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 20:46:42.517087 12786 tablet_service.cc:1467] Processing CreateTablet for tablet 43aec24a3fec44fa836db994de0d0523 (DEFAULT_TABLE table=test-table [id=8a42fb2eba094c16b404b718e7334da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:42.517603 12786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43aec24a3fec44fa836db994de0d0523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:42.518429 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.507216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.5.78.2:40679)
0926 20:46:42.507730 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.507734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.507749 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:46:42.507848 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.507854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.507942 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.508124 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:42.508750 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.510139 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.513748 (+ 3609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.515623 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.517500 (+ 1877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.517652 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.517719 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.517934 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":327,"threads_started":1}
I20260926 20:46:42.519207 12645 tablet_service.cc:1467] Processing CreateTablet for tablet 43aec24a3fec44fa836db994de0d0523 (DEFAULT_TABLE table=test-table [id=8a42fb2eba094c16b404b718e7334da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:42.519665 12645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43aec24a3fec44fa836db994de0d0523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:42.525326 12959 tablet_bootstrap.cc:492] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Bootstrap starting.
I20260926 20:46:42.527112 12960 tablet_bootstrap.cc:492] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Bootstrap starting.
I20260926 20:46:42.527479 12959 tablet_bootstrap.cc:654] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:42.528901 12960 tablet_bootstrap.cc:654] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:42.529928 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.507621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:32775 (local address 127.0.0.1:43916)
0926 20:46:42.508163 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.508182 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:42.508299 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:42.522671 (+ 14372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:42.522677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:42.522709 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:42.523095 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.523113 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.524591 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.524597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.526755 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.526773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.528724 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.528734 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.529007 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.529070 (+ 63us) client_negotiation.cc:769] Sending connection context
0926 20:46:42.529185 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 20:46:42.529393 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":517,"threads_started":1}
I20260926 20:46:42.530781 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.508002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.5.78.1:32775)
0926 20:46:42.518827 (+ 10825us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.518834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.522210 (+ 3376us) server_negotiation.cc:407] Connection header received
0926 20:46:42.522304 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.522312 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.522467 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.523286 (+ 819us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:42.523581 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.524455 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.526962 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.528230 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.528369 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.528524 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.530211 (+ 1687us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.530448 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10686,"thread_start_us":10586,"threads_started":1}
I20260926 20:46:42.531458 12960 tablet_bootstrap.cc:492] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: No bootstrap required, opened a new log
I20260926 20:46:42.531627 12960 ts_tablet_manager.cc:1397] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260926 20:46:42.531572 12500 tablet_service.cc:1467] Processing CreateTablet for tablet 43aec24a3fec44fa836db994de0d0523 (DEFAULT_TABLE table=test-table [id=8a42fb2eba094c16b404b718e7334da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:42.532023 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43aec24a3fec44fa836db994de0d0523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:42.532680 12959 tablet_bootstrap.cc:492] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: No bootstrap required, opened a new log
I20260926 20:46:42.532562 12960 raft_consensus.cc:348] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.532843 12959 ts_tablet_manager.cc:1397] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 20:46:42.532835 12960 raft_consensus.cc:374] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:42.532940 12960 raft_consensus.cc:729] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dce56cb48f4dbea72fec3593fe18fe, State: Initialized, Role: FOLLOWER
I20260926 20:46:42.533231 12960 consensus_queue.cc:260] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.533828 12959 raft_consensus.cc:348] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.534183 12959 raft_consensus.cc:374] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:42.534307 12959 raft_consensus.cc:729] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb3ffe654b54778b4d0b4eb7d9ecbbc, State: Initialized, Role: FOLLOWER
I20260926 20:46:42.534675 12959 consensus_queue.cc:260] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.542600 12960 ts_tablet_manager.cc:1428] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Time spent starting tablet: real 0.011s user 0.002s sys 0.009s
I20260926 20:46:42.544818 12964 raft_consensus.cc:482] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:42.545063 12964 raft_consensus.cc:504] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.544899 12959 ts_tablet_manager.cc:1428] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Time spent starting tablet: real 0.012s user 0.006s sys 0.003s
I20260926 20:46:42.547145 12966 tablet_bootstrap.cc:492] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Bootstrap starting.
I20260926 20:46:42.548434 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.546965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:49571)
0926 20:46:42.547504 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.547645 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"threads_started":1}
I20260926 20:46:42.549273 12966 tablet_bootstrap.cc:654] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:42.550602 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.549582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:40465)
0926 20:46:42.550311 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.550401 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619}
I20260926 20:46:42.551007 12712 heartbeater.cc:503] Master 127.5.78.61:45599 was elected leader, sending a full tablet report...
I20260926 20:46:42.551858 12853 heartbeater.cc:503] Master 127.5.78.61:45599 was elected leader, sending a full tablet report...
I20260926 20:46:42.552179 12964 leader_election.cc:288] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679), e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:42.554359 12966 tablet_bootstrap.cc:492] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: No bootstrap required, opened a new log
I20260926 20:46:42.554586 12966 ts_tablet_manager.cc:1397] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20260926 20:46:42.555526 12966 raft_consensus.cc:348] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.555816 12966 raft_consensus.cc:374] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:42.555981 12966 raft_consensus.cc:729] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f2e1e5187f475ba35eeec7369b743c, State: Initialized, Role: FOLLOWER
I20260926 20:46:42.556265 12966 consensus_queue.cc:260] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.557199 12966 ts_tablet_manager.cc:1428] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 20:46:42.558518 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.557559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:54821)
0926 20:46:42.558064 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.558219 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":399,"threads_started":1}
I20260926 20:46:42.559333 12567 heartbeater.cc:503] Master 127.5.78.61:45599 was elected leader, sending a full tablet report...
I20260926 20:46:42.560309 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.559877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:51161)
0926 20:46:42.560052 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.560149 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 20:46:42.562361 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.551536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:40037 (local address 127.5.78.1:32775)
0926 20:46:42.551749 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.551755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.552426 (+ 671us) server_negotiation.cc:407] Connection header received
0926 20:46:42.552581 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.552589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.552688 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.552808 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:42.554567 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.555814 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.559865 (+ 4051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.561242 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.561335 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.561416 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.561992 (+ 576us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.562119 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 20:46:42.562615 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.551304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:32775 (local address 127.5.78.3:40037)
0926 20:46:42.552265 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.552282 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:42.552428 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:42.552990 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:42.552997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:42.553023 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:42.553319 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.553328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.557348 (+ 4020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.557355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.559633 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.559647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.561556 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.561561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.561708 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.561732 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 20:46:42.561974 (+ 242us) client_negotiation.cc:240] Negotiation successful
0926 20:46:42.562120 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":312,"thread_start_us":266,"threads_started":1}
I20260926 20:46:42.563282 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aec24a3fec44fa836db994de0d0523" candidate_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f2e1e5187f475ba35eeec7369b743c" is_pre_election: true
I20260926 20:46:42.563503 12520 raft_consensus.cc:2454] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cb3ffe654b54778b4d0b4eb7d9ecbbc in term 0.
I20260926 20:46:42.564010 12742 leader_election.cc:302] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb3ffe654b54778b4d0b4eb7d9ecbbc, e0f2e1e5187f475ba35eeec7369b743c; no voters:
I20260926 20:46:42.564442 12964 raft_consensus.cc:2781] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:42.564550 12964 raft_consensus.cc:482] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:42.564656 12964 raft_consensus.cc:3037] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:42.567140 12964 raft_consensus.cc:504] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.567356 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.548152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:38757)
0926 20:46:42.566893 (+ 18741us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.567044 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18601,"thread_start_us":18461,"threads_started":1}
I20260926 20:46:42.568018 12964 leader_election.cc:288] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [CANDIDATE]: Term 1 election: Requested vote from peers 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679), e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775)
I20260926 20:46:42.568743 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aec24a3fec44fa836db994de0d0523" candidate_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f2e1e5187f475ba35eeec7369b743c"
I20260926 20:46:42.568970 12520 raft_consensus.cc:3037] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:42.571206 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.570731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:57007)
0926 20:46:42.570959 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.571057 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:46:42.571766 12520 raft_consensus.cc:2454] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cb3ffe654b54778b4d0b4eb7d9ecbbc in term 1.
I20260926 20:46:42.573071 12742 leader_election.cc:302] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb3ffe654b54778b4d0b4eb7d9ecbbc, e0f2e1e5187f475ba35eeec7369b743c; no voters:
I20260926 20:46:42.573410 12964 raft_consensus.cc:2781] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:42.573625 12964 raft_consensus.cc:686] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 LEADER]: Becoming Leader. State: Replica: 8cb3ffe654b54778b4d0b4eb7d9ecbbc, State: Running, Role: LEADER
I20260926 20:46:42.574083 12964 consensus_queue.cc:237] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } }
I20260926 20:46:42.574913 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.551288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:40679 (local address 127.5.78.3:36565)
0926 20:46:42.551460 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.551478 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:42.551609 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:42.568644 (+ 17035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:42.568650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:42.568693 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:42.568988 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.569001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.570393 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.570399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.572380 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:42.572481 (+ 101us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.574095 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.574101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:42.574278 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.574304 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 20:46:42.574454 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 20:46:42.574594 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 20:46:42.575295 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.574659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:36955)
0926 20:46:42.574829 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.574953 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 20:46:42.575588 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.568058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:36565 (local address 127.5.78.2:40679)
0926 20:46:42.568284 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.568291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.568314 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:46:42.568395 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.568400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.568482 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.568626 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:42.569408 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.570253 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.572656 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.573871 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.574024 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.574160 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.575224 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.575395 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 20:46:42.576488 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aec24a3fec44fa836db994de0d0523" candidate_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dce56cb48f4dbea72fec3593fe18fe" is_pre_election: true
I20260926 20:46:42.576570 12664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aec24a3fec44fa836db994de0d0523" candidate_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dce56cb48f4dbea72fec3593fe18fe"
I20260926 20:46:42.576924 12665 raft_consensus.cc:2454] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cb3ffe654b54778b4d0b4eb7d9ecbbc in term 0.
I20260926 20:46:42.578184 12196 catalog_manager.cc:5495] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3). New cstate: current_term: 1 leader_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cb3ffe654b54778b4d0b4eb7d9ecbbc" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38237 } health_report { overall_health: HEALTHY } } }
I20260926 20:46:42.597796 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.592899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.5.78.3:38237)
0926 20:46:42.593086 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.593090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.593108 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:46:42.593311 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.593315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.593399 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.593525 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:42.593906 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.594717 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.595965 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.596355 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.596443 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.596775 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:42.597324 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:42.597428 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.597526 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.597591 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:46:42.602185 12664 raft_consensus.cc:3037] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:42.602319 12520 raft_consensus.cc:1261] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Refusing update from remote peer 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:42.603168 12965 consensus_queue.cc:1035] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0f2e1e5187f475ba35eeec7369b743c" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:42.605070 12664 raft_consensus.cc:1261] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 FOLLOWER]: Refusing update from remote peer 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:42.606146 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.605483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:33693)
0926 20:46:42.605829 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.605959 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260926 20:46:42.606120 12965 consensus_queue.cc:1035] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dce56cb48f4dbea72fec3593fe18fe" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 40679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:42.609004 12976 mvcc.cc:205] Tried to move back new op lower bound from 7333706148250677248 to 7333706148148105216. Current Snapshot: MvccSnapshot[applied={T|T < 7333706148250677248}]
I20260926 20:46:42.609946 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.609466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:41967)
0926 20:46:42.609622 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.609728 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 20:46:42.612010 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.611409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.1:47491)
0926 20:46:42.611742 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.611846 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260926 20:46:42.612767 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.611804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:46441)
0926 20:46:42.612434 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.612552 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474}
I20260926 20:46:42.617763 12977 mvcc.cc:205] Tried to move back new op lower bound from 7333706148250677248 to 7333706148148105216. Current Snapshot: MvccSnapshot[applied={T|T < 7333706148250677248}]
I20260926 20:46:42.632704 12196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:42.633028 12196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:42.635970 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.635326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.3:34149)
0926 20:46:42.635685 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.635780 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260926 20:46:42.643292 12196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bebec0aad9d04aed8d8d8f7d6ed48653: Sending DeleteTablet for 3 replicas of tablet 43aec24a3fec44fa836db994de0d0523
I20260926 20:46:42.644244 12786 tablet_service.cc:1514] Processing DeleteTablet for tablet 43aec24a3fec44fa836db994de0d0523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:42 UTC) from {username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:54922
I20260926 20:46:42.645143 12645 tablet_service.cc:1514] Processing DeleteTablet for tablet 43aec24a3fec44fa836db994de0d0523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:42 UTC) from {username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:51010
I20260926 20:46:42.646477 12500 tablet_service.cc:1514] Processing DeleteTablet for tablet 43aec24a3fec44fa836db994de0d0523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:42 UTC) from {username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:43916
I20260926 20:46:42.646636 12785 tablet_service.cc:1514] Processing DeleteTablet for tablet 43aec24a3fec44fa836db994de0d0523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:42 UTC) from {username='oryx', principal='oryx/127.5.78.61@KRBTEST.COM'} at 127.0.0.1:54922
I20260926 20:46:42.648168 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.647633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.5.78.2:43219)
0926 20:46:42.647898 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.647998 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260926 20:46:42.649597 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 12423
I20260926 20:46:42.657641 12985 tablet_replica.cc:330] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: stopping tablet replica
I20260926 20:46:42.657874 12985 raft_consensus.cc:2229] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:42.663110 12988 tablet_replica.cc:330] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: stopping tablet replica
I20260926 20:46:42.663359 12988 raft_consensus.cc:2229] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:42.664505 12988 raft_consensus.cc:2258] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:42.665185 12986 tablet_replica.cc:330] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: stopping tablet replica
I20260926 20:46:42.665702 12985 raft_consensus.cc:2258] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:42.668772 12986 raft_consensus.cc:2229] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:42.669502 12986 raft_consensus.cc:2258] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:42.669912 12985 ts_tablet_manager.cc:1905] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:42.670794 12988 ts_tablet_manager.cc:1905] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:42.670975 12986 ts_tablet_manager.cc:1905] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:42.677625 12986 ts_tablet_manager.cc:1918] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:42.677625 12988 ts_tablet_manager.cc:1918] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:42.677784 12988 log.cc:1196] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/43aec24a3fec44fa836db994de0d0523
I20260926 20:46:42.677822 12985 ts_tablet_manager.cc:1918] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:42.677992 12985 log.cc:1196] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/43aec24a3fec44fa836db994de0d0523
I20260926 20:46:42.678025 12986 log.cc:1196] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/43aec24a3fec44fa836db994de0d0523
I20260926 20:46:42.678287 12988 ts_tablet_manager.cc:1939] T 43aec24a3fec44fa836db994de0d0523 P e0f2e1e5187f475ba35eeec7369b743c: Deleting consensus metadata
I20260926 20:46:42.678524 12985 ts_tablet_manager.cc:1939] T 43aec24a3fec44fa836db994de0d0523 P 8cb3ffe654b54778b4d0b4eb7d9ecbbc: Deleting consensus metadata
I20260926 20:46:42.678709 12986 ts_tablet_manager.cc:1939] T 43aec24a3fec44fa836db994de0d0523 P 65dce56cb48f4dbea72fec3593fe18fe: Deleting consensus metadata
I20260926 20:46:42.679648 12183 catalog_manager.cc:4841] TS e0f2e1e5187f475ba35eeec7369b743c (127.5.78.1:32775): tablet 43aec24a3fec44fa836db994de0d0523 (table test-table [id=8a42fb2eba094c16b404b718e7334da6]) successfully deleted
I20260926 20:46:42.679955 12183 catalog_manager.cc:4841] TS 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237): tablet 43aec24a3fec44fa836db994de0d0523 (table test-table [id=8a42fb2eba094c16b404b718e7334da6]) successfully deleted
W20260926 20:46:42.680323 12183 catalog_manager.cc:4808] TS 8cb3ffe654b54778b4d0b4eb7d9ecbbc (127.5.78.3:38237): delete failed for tablet 43aec24a3fec44fa836db994de0d0523 because the tablet was not found. No further retry: Not found: Tablet not found: 43aec24a3fec44fa836db994de0d0523
I20260926 20:46:42.680733 12180 catalog_manager.cc:4841] TS 65dce56cb48f4dbea72fec3593fe18fe (127.5.78.2:40679): tablet 43aec24a3fec44fa836db994de0d0523 (table test-table [id=8a42fb2eba094c16b404b718e7334da6]) successfully deleted
I20260926 20:46:42.691717 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.653602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.5.78.1:32775)
0926 20:46:42.653827 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.653836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.664173 (+ 10337us) server_negotiation.cc:407] Connection header received
0926 20:46:42.664335 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.664345 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.664461 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.664630 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:42.681066 (+ 16436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.681797 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.683141 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.683531 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.683671 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.686186 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:42.686199 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:42.686214 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:42.686251 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:42.688951 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:42.689618 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:42.689625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:42.689627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:42.689753 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:42.690131 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:42.690141 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:42.690143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:42.690563 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:42.690871 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.691150 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.691360 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:46:42.692438 12560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:42.793402 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 12423
I20260926 20:46:42.810814 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 12577
I20260926 20:46:42.826243 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.811710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.5.78.2:40679)
0926 20:46:42.811915 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.811921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.811944 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:46:42.812070 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.812076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.812155 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.812266 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:42.813588 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.814722 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.815987 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.816670 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.816808 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.819554 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:42.819573 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:42.819594 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:42.819642 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:42.823379 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:42.824062 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:42.824073 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:42.824077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:42.824226 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:42.824562 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:42.824567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:42.824571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:42.825024 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:42.825328 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.825639 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.825855 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:46:42.827037 12705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:42.940588 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.940076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39664)
0926 20:46:42.940284 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.940401 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 20:46:42.942775 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.942271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38333 (local address 127.0.0.1:39668)
0926 20:46:42.942460 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:42.942597 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:38333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 20:46:42.942786 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 12577
I20260926 20:46:42.973738 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 12723
I20260926 20:46:42.991196 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:42.974582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.5.78.3:38237)
0926 20:46:42.974758 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:42.974765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:42.974833 (+ 68us) server_negotiation.cc:407] Connection header received
0926 20:46:42.975075 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:42.975081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:42.975169 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:42.975285 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:42.977184 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.978024 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.980067 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:42.980659 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:42.980806 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:42.984331 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:42.984347 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:42.984363 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:42.984410 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:42.988002 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:42.989017 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:42.989025 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:42.989027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:42.989184 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:42.989624 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:42.989627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:42.989629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:42.990010 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:42.990242 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:42.990647 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 20:46:42.990840 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:46:42.991991 12846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:43.123610 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 12723
I20260926 20:46:43.144896 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 12058
I20260926 20:46:43.146199 12226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:43.270344 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 12058
I20260926 20:46:43.291862 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 12157
I20260926 20:46:43.292971 12416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:43.407359 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 12157
2026-09-26T20:46:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12038 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:43 dist-test-slave-z7j8 krb5kdc[12997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:43 dist-test-slave-z7j8 krb5kdc[12997](info): set up 2 sockets
Sep 26 20:46:43 dist-test-slave-z7j8 krb5kdc[12997](info): commencing operation
krb5kdc: starting...
W20260926 20:46:44.567373 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:44 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab.
Entry for principal oryx/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab.
I20260926 20:46:44.782219 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44175
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab
--principal=oryx/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:45.113508 13010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:45.113842 13010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:45.114004 13010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:45.121373 13010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:45.121497 13010 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 20:46:45.121582 13010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:45.121701 13010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:45.121800 13010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:45.121909 13010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:45.132125 13010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44175
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.62.keytab
--principal=oryx/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:45.133543 13010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:45.134999 13010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:45.147320 13018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:45.147320 13015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:45.147368 13016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:45.148551 13010 server_base.cc:1034] running on GCE node
I20260926 20:46:45.149415 13010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:45.151049 13010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:45.152343 13010 hybrid_clock.cc:648] HybridClock initialized: now 1790455605152271 us; error 49 us; skew 500 ppm
I20260926 20:46:45.152791 13010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:45 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455605, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:45.157088 13010 init.cc:377] Logged in from keytab as oryx/127.5.78.62@KRBTEST.COM (short username oryx)
I20260926 20:46:45.158717 13010 webserver.cc:460] Webserver started at http://127.5.78.62:42049/ using document root <none> and password file <none>
I20260926 20:46:45.159456 13010 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:45.159560 13010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:45.159838 13010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:45.163120 13010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f"
format_stamp: "Formatted at 2026-09-26 20:46:45 on dist-test-slave-z7j8"
server_key: "aabe52da0078e5aacbc165bfe9456e83"
server_key_iv: "ae1e3082526a7a530fba404f8c40653a"
server_key_version: "encryptionkey@0"
I20260926 20:46:45.163939 13010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f"
format_stamp: "Formatted at 2026-09-26 20:46:45 on dist-test-slave-z7j8"
server_key: "aabe52da0078e5aacbc165bfe9456e83"
server_key_iv: "ae1e3082526a7a530fba404f8c40653a"
server_key_version: "encryptionkey@0"
I20260926 20:46:45.169335 13010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:46:45.173172 13025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:45.175022 13010 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260926 20:46:45.175215 13010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f"
format_stamp: "Formatted at 2026-09-26 20:46:45 on dist-test-slave-z7j8"
server_key: "aabe52da0078e5aacbc165bfe9456e83"
server_key_iv: "ae1e3082526a7a530fba404f8c40653a"
server_key_version: "encryptionkey@0"
I20260926 20:46:45.175423 13010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:45.200026 13010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:45.204907 13010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:45.205156 13010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:45.228345 13010 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:44175
I20260926 20:46:45.228374 13086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:44175 every 8 connection(s)
I20260926 20:46:45.229835 13010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:45.233794 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13010
I20260926 20:46:45.234004 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:45.234267 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:45.234840 5432 external_mini_cluster.cc:1428] Setting key 809478f02a52cf80e1eb4f95c36f44a9
I20260926 20:46:45.249033 13087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Bootstrap starting.
Sep 26 20:46:45 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:45.253017 13087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:45.254220 13087 log.cc:824] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:45.256964 13087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: No bootstrap required, opened a new log
I20260926 20:46:45.258831 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:45.236965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.5.78.62:44175)
0926 20:46:45.237860 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:45.237873 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:45.237920 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:46:45.239629 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:45.239655 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:45.240074 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:45.240433 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:45.242207 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.243617 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.245383 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.246131 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.246273 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:45.251138 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:45.251146 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:45.251161 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:45.251201 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:45.254762 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:45.255475 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:45.255483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:45.255492 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:45.255666 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:45.256069 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:45.256075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:45.256079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:45.256583 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:45.256876 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:45.257332 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 20:46:45.257756 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":415,"threads_started":1}
I20260926 20:46:45.264691 13087 raft_consensus.cc:348] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44175 } }
I20260926 20:46:45.265066 13087 raft_consensus.cc:374] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:45.265173 13087 raft_consensus.cc:729] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a12ebe8eab8f4e1ca46663a4b7d2650f, State: Initialized, Role: FOLLOWER
I20260926 20:46:45.265763 13087 consensus_queue.cc:260] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44175 } }
I20260926 20:46:45.266016 13087 raft_consensus.cc:388] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:45.266156 13087 raft_consensus.cc:482] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:45.266331 13087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:45.268921 13087 raft_consensus.cc:504] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44175 } }
I20260926 20:46:45.269435 13087 leader_election.cc:302] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a12ebe8eab8f4e1ca46663a4b7d2650f; no voters:
I20260926 20:46:45.270059 13087 leader_election.cc:288] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:45.270300 13092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:45.271055 13092 raft_consensus.cc:686] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [term 1 LEADER]: Becoming Leader. State: Replica: a12ebe8eab8f4e1ca46663a4b7d2650f, State: Running, Role: LEADER
I20260926 20:46:45.271592 13092 consensus_queue.cc:237] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44175 } }
I20260926 20:46:45.272142 13087 sys_catalog.cc:564] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:45.274627 13094 sys_catalog.cc:455] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a12ebe8eab8f4e1ca46663a4b7d2650f. Latest consensus state: current_term: 1 leader_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44175 } } }
I20260926 20:46:45.274690 13093 sys_catalog.cc:455] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12ebe8eab8f4e1ca46663a4b7d2650f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44175 } } }
I20260926 20:46:45.275027 13094 sys_catalog.cc:458] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:45.275338 13093 sys_catalog.cc:458] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:45.282642 13099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:45.287719 13099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:45.297446 13099 catalog_manager.cc:1347] Generated new cluster ID: 14be6bf8ff814bca8f695174ec21a41d
I20260926 20:46:45.297582 13099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:45.319780 13099 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:45.321673 13099 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:45.331557 13099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Generated new TSK 0
I20260926 20:46:45.332631 13099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab.
Entry for principal oryx/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab.
I20260926 20:46:45.387475 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44175
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:45.728139 13113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:45.728463 13113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:45.728570 13113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:45.736217 13113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:45.736362 13113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:45.736459 13113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:46:45.736550 13113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:45.746658 13113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:44175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:45.747987 13113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:45.749377 13113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:45.763875 13121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:45.763908 13118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:45.763947 13119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:45.764706 13113 server_base.cc:1034] running on GCE node
I20260926 20:46:45.765409 13113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:45.766469 13113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:45.767767 13113 hybrid_clock.cc:648] HybridClock initialized: now 1790455605767695 us; error 48 us; skew 500 ppm
I20260926 20:46:45.768168 13113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:45 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455605, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:45.771601 13113 init.cc:377] Logged in from keytab as oryx/127.5.78.1@KRBTEST.COM (short username oryx)
I20260926 20:46:45.773296 13113 webserver.cc:460] Webserver started at http://127.5.78.1:45241/ using document root <none> and password file <none>
I20260926 20:46:45.774211 13113 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:45.774353 13113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:45.774734 13113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:45.777864 13113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "90f1f7a896dd4c63be76af2920e1366a"
format_stamp: "Formatted at 2026-09-26 20:46:45 on dist-test-slave-z7j8"
server_key: "b58f3ac3c6605aa7c39abb3e8c59fb01"
server_key_iv: "f040a856eeac3e18bf03b0b80df505f1"
server_key_version: "encryptionkey@0"
I20260926 20:46:45.778870 13113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "90f1f7a896dd4c63be76af2920e1366a"
format_stamp: "Formatted at 2026-09-26 20:46:45 on dist-test-slave-z7j8"
server_key: "b58f3ac3c6605aa7c39abb3e8c59fb01"
server_key_iv: "f040a856eeac3e18bf03b0b80df505f1"
server_key_version: "encryptionkey@0"
I20260926 20:46:45.784746 13113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:45.788728 13128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:45.790258 13113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:46:45.790544 13113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "90f1f7a896dd4c63be76af2920e1366a"
format_stamp: "Formatted at 2026-09-26 20:46:45 on dist-test-slave-z7j8"
server_key: "b58f3ac3c6605aa7c39abb3e8c59fb01"
server_key_iv: "f040a856eeac3e18bf03b0b80df505f1"
server_key_version: "encryptionkey@0"
I20260926 20:46:45.790814 13113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:45.813360 13113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:45.818148 13113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:45.818548 13113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:45.821429 13113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:45.821558 13113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:45.821725 13113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:45.821810 13113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:46:45 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455605, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:45.848984 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:45.822304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:45523 (local address 127.5.78.62:44175)
0926 20:46:45.824375 (+ 2071us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:45.824381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:45.824404 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:46:45.826263 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:45.826270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:45.826381 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:45.826545 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:45.828822 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.830161 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.833214 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.834076 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.834317 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:45.839874 (+ 5557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:45.839886 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:45.839894 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:45.839966 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:45.844031 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:45.845467 (+ 1436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:45.845474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:45.845477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:45.845636 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:45.846427 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:45.846435 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:45.846441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:45.846798 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:45.846966 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:45.848076 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0926 20:46:45.848428 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1954,"thread_start_us":1860,"threads_started":1}
I20260926 20:46:45.849712 13138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:45.822922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.5.78.1:45523)
0926 20:46:45.823849 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:45.823899 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:45.825840 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:45.826820 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:45.826838 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:45.827758 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:45.828601 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:45.828622 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.830375 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.830382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:45.833013 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:45.833037 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.834384 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.834390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:45.834822 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:45.835659 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:45.835706 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:45.839445 (+ 3739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:45.844289 (+ 4844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:45.844302 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:45.844329 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:45.845241 (+ 912us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:45.845834 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:45.845841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:45.845847 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:45.846217 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:45.846945 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:45.846953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:45.847498 (+ 545us) client_negotiation.cc:769] Sending connection context
0926 20:46:45.847904 (+ 406us) client_negotiation.cc:240] Negotiation successful
0926 20:46:45.848377 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":427,"threads_started":1}
I20260926 20:46:45.872365 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:45.857211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:40715 (local address 127.5.78.62:44175)
0926 20:46:45.857455 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:45.857465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:45.858019 (+ 554us) server_negotiation.cc:407] Connection header received
0926 20:46:45.858194 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:45.858202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:45.858310 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:45.858478 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:45.859423 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.860879 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.862232 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.863060 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.863214 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:45.866121 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:45.866130 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:45.866138 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:45.866186 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:45.869014 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:45.870104 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:45.870108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:45.870110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:45.870237 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:45.870699 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:45.870702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:45.870703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:45.870974 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:45.871085 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:45.871671 (+ 586us) server_negotiation.cc:299] Negotiation successful
0926 20:46:45.871957 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:46:45.872915 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:45.857209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.5.78.1:40715)
0926 20:46:45.857916 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:45.857932 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:45.858056 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:45.858537 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:45.858540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:45.858989 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:45.859275 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:45.859284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.861060 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.861064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:45.862080 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:45.862093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:45.863512 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:45.863519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:45.863783 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:45.864465 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:45.864495 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:45.865875 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:45.869206 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:45.869215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:45.869222 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:45.869965 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:45.870366 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:45.870374 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:45.870380 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:45.870591 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:45.871106 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:45.871114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:45.871399 (+ 285us) client_negotiation.cc:769] Sending connection context
0926 20:46:45.871596 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 20:46:45.872219 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":375,"threads_started":1}
I20260926 20:46:45.878764 13113 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:43657
I20260926 20:46:45.878784 13248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:43657 every 8 connection(s)
I20260926 20:46:45.880040 13113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:45.882277 13249 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44175
I20260926 20:46:45.882557 13249 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:45.883291 13249 heartbeater.cc:511] Master 127.5.78.62:44175 requested a full tablet report, sending...
I20260926 20:46:45.885490 13042 ts_manager.cc:194] Registered new tserver with Master: 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657)
I20260926 20:46:45.887076 13042 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:45523
I20260926 20:46:45.890519 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13113
I20260926 20:46:45.890712 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:45.891353 5432 external_mini_cluster.cc:1428] Setting key 9fa510e9ec4a708de9b09114a673d12b
WARNING: no policy specified for oryx/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab.
Entry for principal oryx/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab.
I20260926 20:46:45.941385 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44175
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:46.270105 13253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:46.270489 13253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:46.270622 13253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:46.278231 13253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:46.278365 13253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:46.278559 13253 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:46:46.278692 13253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:46.288933 13253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:44175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:46.290280 13253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:46.291728 13253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:46.305676 13258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:46.305687 13261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:46.305932 13259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:46.306527 13253 server_base.cc:1034] running on GCE node
I20260926 20:46:46.307196 13253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:46.308180 13253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:46.309422 13253 hybrid_clock.cc:648] HybridClock initialized: now 1790455606309363 us; error 44 us; skew 500 ppm
I20260926 20:46:46.309751 13253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:46 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455606, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:46.312858 13253 init.cc:377] Logged in from keytab as oryx/127.5.78.2@KRBTEST.COM (short username oryx)
I20260926 20:46:46.314397 13253 webserver.cc:460] Webserver started at http://127.5.78.2:33535/ using document root <none> and password file <none>
I20260926 20:46:46.315173 13253 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:46.315302 13253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:46.315644 13253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:46.318753 13253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "ec2d4e81fe364ff2ae8dd30154f97988"
format_stamp: "Formatted at 2026-09-26 20:46:46 on dist-test-slave-z7j8"
server_key: "fdcc90c2d5a290db37d945ca80890d19"
server_key_iv: "5b65cfa10ed70983773088e2f2330f65"
server_key_version: "encryptionkey@0"
I20260926 20:46:46.319586 13253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "ec2d4e81fe364ff2ae8dd30154f97988"
format_stamp: "Formatted at 2026-09-26 20:46:46 on dist-test-slave-z7j8"
server_key: "fdcc90c2d5a290db37d945ca80890d19"
server_key_iv: "5b65cfa10ed70983773088e2f2330f65"
server_key_version: "encryptionkey@0"
I20260926 20:46:46.324965 13253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:46.328617 13268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:46.330065 13253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:46:46.330256 13253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "ec2d4e81fe364ff2ae8dd30154f97988"
format_stamp: "Formatted at 2026-09-26 20:46:46 on dist-test-slave-z7j8"
server_key: "fdcc90c2d5a290db37d945ca80890d19"
server_key_iv: "5b65cfa10ed70983773088e2f2330f65"
server_key_version: "encryptionkey@0"
I20260926 20:46:46.330578 13253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:46.368057 13253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:46.372826 13253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:46.373119 13253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:46.376138 13253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:46.376252 13253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:46.376403 13253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:46.376483 13253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:46:46 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455606, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:46.404181 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.376646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:43983 (local address 127.5.78.62:44175)
0926 20:46:46.384107 (+ 7461us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:46.384112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:46.384129 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:46:46.384182 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:46.384185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:46.384264 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:46.384398 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:46.386250 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.387386 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.390484 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.391179 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.391345 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.395829 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:46.395835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:46.395838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:46.395901 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:46.399441 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.400640 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.400646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.400650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.400833 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.401580 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.401587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.401593 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.401964 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:46.402123 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:46.403247 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0926 20:46:46.403593 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7348,"thread_start_us":7258,"threads_started":1}
I20260926 20:46:46.404763 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.377441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.5.78.2:43983)
0926 20:46:46.379268 (+ 1827us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:46.379346 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:46.383264 (+ 3918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:46.384684 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:46.384694 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:46.385308 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:46.386020 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.386041 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.387601 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.387610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.390226 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.390255 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.391462 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.391470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.391704 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.392477 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:46.392504 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:46.395481 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:46.399616 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.399631 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.399658 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.400425 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.401022 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.401034 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.401041 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.401425 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.402121 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:46.402131 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:46.402624 (+ 493us) client_negotiation.cc:769] Sending connection context
0926 20:46:46.403032 (+ 408us) client_negotiation.cc:240] Negotiation successful
0926 20:46:46.403504 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":434,"threads_started":1}
I20260926 20:46:46.426874 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.411170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:57007 (local address 127.5.78.62:44175)
0926 20:46:46.411403 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:46.411413 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:46.412088 (+ 675us) server_negotiation.cc:407] Connection header received
0926 20:46:46.412247 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:46.412263 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:46.412378 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:46.412520 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:46.413471 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.414899 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.416955 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.417372 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.417515 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.420720 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:46.420730 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:46.420734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:46.420783 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:46.424085 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.424823 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.424830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.424834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.424962 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.425353 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.425361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.425366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.425693 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:46.425846 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:46.426262 (+ 416us) server_negotiation.cc:299] Negotiation successful
0926 20:46:46.426524 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:46:46.427042 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.411321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.5.78.2:57007)
0926 20:46:46.411969 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:46.411995 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:46.412121 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:46.412551 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:46.412554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:46.413030 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:46.413319 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.413329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.415064 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.415075 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.416798 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.416811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.417588 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.417594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.417874 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.418779 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:46.418807 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:46.420453 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:46.424239 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.424246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.424250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.424679 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.425082 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.425087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.425091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.425220 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.425823 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:46.425827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:46.426013 (+ 186us) client_negotiation.cc:769] Sending connection context
0926 20:46:46.426162 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 20:46:46.426537 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":357,"threads_started":1}
I20260926 20:46:46.437265 13253 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:36179
I20260926 20:46:46.437314 13388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:36179 every 8 connection(s)
I20260926 20:46:46.438593 13253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:46.440663 13389 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44175
I20260926 20:46:46.440892 13389 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:46.441517 13389 heartbeater.cc:511] Master 127.5.78.62:44175 requested a full tablet report, sending...
I20260926 20:46:46.442960 13042 ts_manager.cc:194] Registered new tserver with Master: ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179)
I20260926 20:46:46.444047 13042 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:43983
I20260926 20:46:46.444089 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13253
I20260926 20:46:46.444275 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:46.444864 5432 external_mini_cluster.cc:1428] Setting key d7e6bae8ff88baf11df36fe0aaa32733
WARNING: no policy specified for oryx/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab.
Entry for principal oryx/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab.
I20260926 20:46:46.491326 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:44175
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:46.819636 13393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:46.819943 13393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:46.820120 13393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:46.827411 13393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:46.827518 13393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:46.827694 13393 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:46:46.827836 13393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:46.837884 13393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.78.62:44175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:46.839318 13393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:46.840672 13393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:46.853894 13399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:46.854107 13398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:46.854502 13401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:46.854894 13393 server_base.cc:1034] running on GCE node
I20260926 20:46:46.855571 13393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:46.856506 13393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:46.857748 13393 hybrid_clock.cc:648] HybridClock initialized: now 1790455606857679 us; error 50 us; skew 500 ppm
I20260926 20:46:46.858054 13393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:46 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455606, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:46.861124 13393 init.cc:377] Logged in from keytab as oryx/127.5.78.3@KRBTEST.COM (short username oryx)
I20260926 20:46:46.862560 13393 webserver.cc:460] Webserver started at http://127.5.78.3:34135/ using document root <none> and password file <none>
I20260926 20:46:46.863214 13393 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:46.863307 13393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:46.863569 13393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:46.866493 13393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "18b20c16f9d242f9b1dd570d171f390b"
format_stamp: "Formatted at 2026-09-26 20:46:46 on dist-test-slave-z7j8"
server_key: "572a3e41d9f47624b9eab00df90618f0"
server_key_iv: "17eb87f1bd0046c167e4e0ff219380e8"
server_key_version: "encryptionkey@0"
I20260926 20:46:46.867256 13393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "18b20c16f9d242f9b1dd570d171f390b"
format_stamp: "Formatted at 2026-09-26 20:46:46 on dist-test-slave-z7j8"
server_key: "572a3e41d9f47624b9eab00df90618f0"
server_key_iv: "17eb87f1bd0046c167e4e0ff219380e8"
server_key_version: "encryptionkey@0"
I20260926 20:46:46.872407 13393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:46.876441 13408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:46.878139 13393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:46:46.878314 13393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "18b20c16f9d242f9b1dd570d171f390b"
format_stamp: "Formatted at 2026-09-26 20:46:46 on dist-test-slave-z7j8"
server_key: "572a3e41d9f47624b9eab00df90618f0"
server_key_iv: "17eb87f1bd0046c167e4e0ff219380e8"
server_key_version: "encryptionkey@0"
I20260926 20:46:46.878639 13393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:46.889967 13249 heartbeater.cc:503] Master 127.5.78.62:44175 was elected leader, sending a full tablet report...
I20260926 20:46:46.912629 13393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:46.917193 13393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:46.917503 13393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:46.920406 13393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:46.920493 13393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:46.920661 13393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:46.920722 13393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:46:46 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455606, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:46.946872 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.921224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:36715 (local address 127.5.78.62:44175)
0926 20:46:46.921418 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:46.921424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:46.922990 (+ 1566us) server_negotiation.cc:407] Connection header received
0926 20:46:46.925308 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:46.925314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:46.925411 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:46.925527 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:46.928501 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.929529 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.932969 (+ 3440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.933552 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.933710 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.939676 (+ 5966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:46.939685 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:46.939693 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:46.939745 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:46.942544 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.943593 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.943601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.943608 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.943799 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.944386 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.944391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.944395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.944821 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:46.944995 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:46.946244 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0926 20:46:46.946569 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 20:46:46.948961 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.921850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.5.78.3:36715)
0926 20:46:46.922785 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:46.922833 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:46.924897 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:46.925899 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:46.925918 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:46.927044 (+ 1126us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:46.928225 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.928265 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.929726 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.929735 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.932755 (+ 3020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.932779 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.933861 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.933872 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.934255 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.935353 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:46.935393 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:46.939351 (+ 3958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:46.942696 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.942706 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.942729 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.943418 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.943925 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.943932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.943938 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.944253 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.944994 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:46.945010 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:46.945664 (+ 654us) client_negotiation.cc:769] Sending connection context
0926 20:46:46.946096 (+ 432us) client_negotiation.cc:240] Negotiation successful
0926 20:46:46.946871 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":407,"threads_started":1}
I20260926 20:46:46.973130 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.957496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:38845 (local address 127.5.78.62:44175)
0926 20:46:46.957755 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:46.957765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:46.958380 (+ 615us) server_negotiation.cc:407] Connection header received
0926 20:46:46.958651 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:46.958660 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:46.958774 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:46.958906 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:46.959932 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.961439 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.963033 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.963823 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.963984 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.966375 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:46.966382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:46.966388 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:46.966456 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:46.969925 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.970803 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.970809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.970812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.970978 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:46.971536 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:46.971544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:46.971548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:46.971963 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:46.972118 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:46.972528 (+ 410us) server_negotiation.cc:299] Negotiation successful
0926 20:46:46.972739 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:46:46.973263 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:46.957447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.5.78.3:38845)
0926 20:46:46.958227 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:46.958253 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:46.958491 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:46.958957 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:46.958960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:46.959401 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:46.959765 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.959783 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.961617 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.961621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.962801 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:46.962823 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:46.964039 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:46.964046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:46.964185 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:46.964940 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:46.964973 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:46.966135 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:46.970059 (+ 3924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.970067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.970073 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.970665 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.971154 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:46.971160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:46.971171 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:46.971379 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:46.972084 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:46.972089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:46.972273 (+ 184us) client_negotiation.cc:769] Sending connection context
0926 20:46:46.972419 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 20:46:46.972749 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":394,"threads_started":1}
I20260926 20:46:46.977712 13393 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:34741
I20260926 20:46:46.977731 13527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:34741 every 8 connection(s)
I20260926 20:46:46.979055 13393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:46.981312 13528 heartbeater.cc:347] Connected to a master server at 127.5.78.62:44175
I20260926 20:46:46.981544 13528 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:46.982173 13528 heartbeater.cc:511] Master 127.5.78.62:44175 requested a full tablet report, sending...
I20260926 20:46:46.983493 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13393
I20260926 20:46:46.983695 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:46.983878 13042 ts_manager.cc:194] Registered new tserver with Master: 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:46.984175 5432 external_mini_cluster.cc:1428] Setting key 7d00146bf3de5c0e93c09a27d32c32da
I20260926 20:46:46.984973 13042 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:36715
I20260926 20:46:46.987335 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:46:47 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:46:47 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:47 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:46:47.037961 13277 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:47.027056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.5.78.62:44175)
0926 20:46:47.027291 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.027298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.027551 (+ 253us) server_negotiation.cc:407] Connection header received
0926 20:46:47.027698 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.027702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.027765 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.027857 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:47.029256 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.030000 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.031306 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.031735 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.031839 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.037681 (+ 5842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34962: BlockingRecv error: recv got EOF from 127.0.0.1:34962 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 20:46:47 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:47.055555 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.041149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.5.78.62:44175)
0926 20:46:47.041391 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.041400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.041664 (+ 264us) server_negotiation.cc:407] Connection header received
0926 20:46:47.041874 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.041880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.041956 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.042061 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:47.043235 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.043998 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.045208 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.045651 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.045796 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.049645 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:47.049650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:47.049653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:47.049688 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:47.052603 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:47.053474 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:47.053487 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:47.053490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:47.053623 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:47.054042 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:47.054047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:47.054050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:47.054370 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:47.054684 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.054989 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.055222 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:46:47.059682 13042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:47.067075 13042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:47.091014 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.080757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.5.78.3:34741)
0926 20:46:47.081437 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.081443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.082163 (+ 720us) server_negotiation.cc:407] Connection header received
0926 20:46:47.082522 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.082532 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.082848 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.083062 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.084209 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.085548 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.088741 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.089692 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.089860 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.090108 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.090459 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.090662 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":361,"threads_started":1}
I20260926 20:46:47.091301 13546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.080542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:34741 (local address 127.0.0.1:60064)
0926 20:46:47.082007 (+ 1465us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.082052 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.082237 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.083416 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.083423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.083450 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.083979 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.083993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.085688 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.085696 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.088523 (+ 2827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.088535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.089889 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.089894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.090155 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.090228 (+ 73us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.090398 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.090841 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"mutex_wait_us":160,"thread_start_us":1091,"threads_started":1}
I20260926 20:46:47.095422 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.082521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:36179 (local address 127.0.0.1:38076)
0926 20:46:47.083223 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.083245 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.083433 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.087368 (+ 3935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.087372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.087396 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.087969 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.087987 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.089972 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.089978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.092684 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.092698 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.094509 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.094520 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.094683 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.094713 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.094838 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.094967 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":74,"thread_start_us":323,"threads_started":1}
I20260926 20:46:47.095638 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.082798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.5.78.2:36179)
0926 20:46:47.086892 (+ 4094us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.086899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.086921 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:46:47.086999 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.087004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.087181 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.087387 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.088257 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.089778 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.092843 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.094273 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.094400 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.094630 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.095110 (+ 480us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.095357 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3935,"thread_start_us":3807,"threads_started":1}
I20260926 20:46:47.095894 13551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.084107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43657 (local address 127.0.0.1:58848)
0926 20:46:47.086600 (+ 2493us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.086623 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.086735 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.087580 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.087585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.087613 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.088236 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.088246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.090000 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.090006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.092623 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.092639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.094587 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.094593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.094757 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.094823 (+ 66us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.094906 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.095006 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2332,"mutex_wait_us":121,"thread_start_us":2161,"threads_started":1}
I20260926 20:46:47.096418 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.083608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.5.78.1:43657)
0926 20:46:47.086109 (+ 2501us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.086120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.086866 (+ 746us) server_negotiation.cc:407] Connection header received
0926 20:46:47.086962 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.086970 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.087213 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.087386 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.088413 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.089828 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.092819 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.094273 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.095164 (+ 891us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.095370 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.095445 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.096166 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2319,"thread_start_us":2171,"threads_started":1}
I20260926 20:46:47.095618 13458 tablet_service.cc:1467] Processing CreateTablet for tablet 72b026ef2ccc4e6cb0ab581b86f24c98 (DEFAULT_TABLE table=test-table [id=5cf1e0353c0b47b8b0b18b4a78a43b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:47.097527 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b026ef2ccc4e6cb0ab581b86f24c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.099738 13179 tablet_service.cc:1467] Processing CreateTablet for tablet 72b026ef2ccc4e6cb0ab581b86f24c98 (DEFAULT_TABLE table=test-table [id=5cf1e0353c0b47b8b0b18b4a78a43b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:47.099551 13319 tablet_service.cc:1467] Processing CreateTablet for tablet 72b026ef2ccc4e6cb0ab581b86f24c98 (DEFAULT_TABLE table=test-table [id=5cf1e0353c0b47b8b0b18b4a78a43b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:47.101006 13179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b026ef2ccc4e6cb0ab581b86f24c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.101020 13319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b026ef2ccc4e6cb0ab581b86f24c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.112550 13552 tablet_bootstrap.cc:492] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Bootstrap starting.
I20260926 20:46:47.115132 13553 tablet_bootstrap.cc:492] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Bootstrap starting.
I20260926 20:46:47.116209 13552 tablet_bootstrap.cc:654] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.117547 13552 log.cc:824] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:47.118764 13553 tablet_bootstrap.cc:654] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.120095 13553 log.cc:824] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:47.121183 13552 tablet_bootstrap.cc:492] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: No bootstrap required, opened a new log
I20260926 20:46:47.121469 13552 ts_tablet_manager.cc:1397] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:46:47.122498 13554 tablet_bootstrap.cc:492] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Bootstrap starting.
I20260926 20:46:47.123104 13553 tablet_bootstrap.cc:492] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: No bootstrap required, opened a new log
I20260926 20:46:47.123360 13553 ts_tablet_manager.cc:1397] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 20:46:47.125310 13554 tablet_bootstrap.cc:654] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.126358 13554 log.cc:824] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:47.128997 13554 tablet_bootstrap.cc:492] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: No bootstrap required, opened a new log
I20260926 20:46:47.129278 13554 ts_tablet_manager.cc:1397] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260926 20:46:47.132277 13552 raft_consensus.cc:348] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.132757 13552 raft_consensus.cc:374] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.132916 13552 raft_consensus.cc:729] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.133591 13552 consensus_queue.cc:260] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.133899 13553 raft_consensus.cc:348] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.134428 13553 raft_consensus.cc:374] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.134584 13553 raft_consensus.cc:729] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b20c16f9d242f9b1dd570d171f390b, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.135315 13553 consensus_queue.cc:260] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.135728 13552 ts_tablet_manager.cc:1428] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 20:46:47.136650 13528 heartbeater.cc:503] Master 127.5.78.62:44175 was elected leader, sending a full tablet report...
I20260926 20:46:47.137333 13553 ts_tablet_manager.cc:1428] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260926 20:46:47.138971 13554 raft_consensus.cc:348] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.139382 13554 raft_consensus.cc:374] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.139510 13554 raft_consensus.cc:729] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.140233 13554 consensus_queue.cc:260] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.141641 13389 heartbeater.cc:503] Master 127.5.78.62:44175 was elected leader, sending a full tablet report...
I20260926 20:46:47.142122 13554 ts_tablet_manager.cc:1428] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260926 20:46:47.190796 13390 tablet.cc:2369] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:46:47.230734 13529 tablet.cc:2369] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:46:47.322464 13558 raft_consensus.cc:482] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:47.322783 13558 raft_consensus.cc:504] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.324488 13558 leader_election.cc:288] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741), ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179)
I20260926 20:46:47.331800 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.325136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:33387 (local address 127.5.78.3:34741)
0926 20:46:47.325367 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.325371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.325877 (+ 506us) server_negotiation.cc:407] Connection header received
0926 20:46:47.326162 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.326173 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.326263 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.326325 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.327092 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.328137 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.330041 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.330880 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.331039 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.331150 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.331429 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.331548 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:46:47.331889 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.326259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:51905 (local address 127.5.78.2:36179)
0926 20:46:47.326521 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.326529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.326683 (+ 154us) server_negotiation.cc:407] Connection header received
0926 20:46:47.326924 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.326929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.326996 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.327102 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.327518 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.328449 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.330179 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.330964 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.331124 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.331237 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.331515 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.331639 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:46:47.331997 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.324985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:34741 (local address 127.5.78.1:33387)
0926 20:46:47.325745 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.325766 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.325927 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.326582 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.326586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.326608 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.326946 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.326954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.328270 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.328274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.329846 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.329859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.331100 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.331106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.331258 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.331281 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.331385 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.331523 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":31,"spinlock_wait_cycles":32640,"thread_start_us":424,"threads_started":1}
I20260926 20:46:47.332041 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.325907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:36179 (local address 127.5.78.1:51905)
0926 20:46:47.326560 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.326583 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.326783 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.327103 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.327105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.327122 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.327379 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.327385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.328602 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.328607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.330024 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.330036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.331198 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.331204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.331358 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.331380 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.331481 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.331602 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":22,"thread_start_us":297,"threads_started":1}
I20260926 20:46:47.333150 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b026ef2ccc4e6cb0ab581b86f24c98" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" is_pre_election: true
I20260926 20:46:47.333150 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b026ef2ccc4e6cb0ab581b86f24c98" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b" is_pre_election: true
I20260926 20:46:47.333650 13478 raft_consensus.cc:2454] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 0.
I20260926 20:46:47.333649 13344 raft_consensus.cc:2454] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 0.
I20260926 20:46:47.334363 13131 leader_election.cc:302] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b20c16f9d242f9b1dd570d171f390b, 90f1f7a896dd4c63be76af2920e1366a; no voters:
I20260926 20:46:47.334875 13558 raft_consensus.cc:2781] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:47.335000 13558 raft_consensus.cc:482] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:47.335120 13558 raft_consensus.cc:3037] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.337321 13558 raft_consensus.cc:504] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.337918 13558 leader_election.cc:288] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 election: Requested vote from peers 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741), ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179)
I20260926 20:46:47.338593 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b026ef2ccc4e6cb0ab581b86f24c98" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b"
I20260926 20:46:47.338657 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b026ef2ccc4e6cb0ab581b86f24c98" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2d4e81fe364ff2ae8dd30154f97988"
I20260926 20:46:47.338817 13478 raft_consensus.cc:3037] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.338934 13344 raft_consensus.cc:3037] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.341123 13478 raft_consensus.cc:2454] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 1.
I20260926 20:46:47.341368 13344 raft_consensus.cc:2454] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 1.
I20260926 20:46:47.341653 13131 leader_election.cc:302] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b20c16f9d242f9b1dd570d171f390b, 90f1f7a896dd4c63be76af2920e1366a; no voters:
I20260926 20:46:47.342006 13558 raft_consensus.cc:2781] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:47.342576 13558 raft_consensus.cc:686] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 1 LEADER]: Becoming Leader. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Running, Role: LEADER
I20260926 20:46:47.343225 13558 consensus_queue.cc:237] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } }
I20260926 20:46:47.347965 13041 catalog_manager.cc:5495] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1). New cstate: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } health_report { overall_health: HEALTHY } } }
W20260926 20:46:47.382781 13250 tablet.cc:2369] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:46:47.389555 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.383943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.5.78.1:43657)
0926 20:46:47.384174 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.384181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.384268 (+ 87us) server_negotiation.cc:407] Connection header received
0926 20:46:47.384549 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.384553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.384638 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.384805 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:47.385177 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.385893 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.387381 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.387802 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.387941 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.388385 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:47.388961 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:47.389137 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.389190 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.389275 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:46:47.398102 13344 raft_consensus.cc:1261] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Refusing update from remote peer 90f1f7a896dd4c63be76af2920e1366a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:47.398288 13478 raft_consensus.cc:1261] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Refusing update from remote peer 90f1f7a896dd4c63be76af2920e1366a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:47.399510 13563 consensus_queue.cc:1035] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:47.400003 13558 consensus_queue.cc:1035] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:47.413235 13568 mvcc.cc:205] Tried to move back new op lower bound from 7333706167892131840 to 7333706167681916928. Current Snapshot: MvccSnapshot[applied={T|T < 7333706167892131840}]
I20260926 20:46:47.427011 13041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:47.427316 13041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:47.431171 13041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Sending DeleteTablet for 3 replicas of tablet 72b026ef2ccc4e6cb0ab581b86f24c98
I20260926 20:46:47.432473 13458 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b026ef2ccc4e6cb0ab581b86f24c98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:47 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:60064
I20260926 20:46:47.432634 13319 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b026ef2ccc4e6cb0ab581b86f24c98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:47 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:38076
I20260926 20:46:47.433597 13179 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b026ef2ccc4e6cb0ab581b86f24c98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:47 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:58848
I20260926 20:46:47.434450 13042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:47.435704 13042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:47.447065 13178 tablet_service.cc:1467] Processing CreateTablet for tablet 5f09d669769647a6b599e197535ad215 (DEFAULT_TABLE table=test-table [id=7b30c21959c7493f8f41933f90788901]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:47.447950 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f09d669769647a6b599e197535ad215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.450977 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 5f09d669769647a6b599e197535ad215 (DEFAULT_TABLE table=test-table [id=7b30c21959c7493f8f41933f90788901]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:47.451565 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f09d669769647a6b599e197535ad215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.453526 13577 tablet_replica.cc:330] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: stopping tablet replica
I20260926 20:46:47.454109 13577 raft_consensus.cc:2229] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:47.454766 13577 raft_consensus.cc:2258] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:47.458190 13553 tablet_bootstrap.cc:492] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Bootstrap starting.
I20260926 20:46:47.459874 13553 tablet_bootstrap.cc:654] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.463562 13578 tablet_replica.cc:330] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: stopping tablet replica
I20260926 20:46:47.464121 13578 raft_consensus.cc:2229] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:47.464862 13578 raft_consensus.cc:2258] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:47.467468 13552 tablet_bootstrap.cc:492] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Bootstrap starting.
I20260926 20:46:47.467876 13576 tablet_replica.cc:330] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: stopping tablet replica
I20260926 20:46:47.469115 13552 tablet_bootstrap.cc:654] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.468351 13576 raft_consensus.cc:2229] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:47.470127 13576 raft_consensus.cc:2258] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:47.454717 13318 tablet_service.cc:1467] Processing CreateTablet for tablet 5f09d669769647a6b599e197535ad215 (DEFAULT_TABLE table=test-table [id=7b30c21959c7493f8f41933f90788901]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:47.470709 13553 tablet_bootstrap.cc:492] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: No bootstrap required, opened a new log
I20260926 20:46:47.470928 13553 ts_tablet_manager.cc:1397] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260926 20:46:47.471211 13318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f09d669769647a6b599e197535ad215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.471753 13553 raft_consensus.cc:348] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.472090 13553 raft_consensus.cc:374] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.472215 13553 raft_consensus.cc:729] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b20c16f9d242f9b1dd570d171f390b, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.472529 13553 consensus_queue.cc:260] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.473197 13578 ts_tablet_manager.cc:1905] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:47.475719 13553 ts_tablet_manager.cc:1428] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260926 20:46:47.480681 13554 tablet_bootstrap.cc:492] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Bootstrap starting.
I20260926 20:46:47.481102 13576 ts_tablet_manager.cc:1905] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:47.482504 13554 tablet_bootstrap.cc:654] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.483968 13577 ts_tablet_manager.cc:1905] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:47.484298 13578 ts_tablet_manager.cc:1918] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:47.484552 13578 log.cc:1196] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/72b026ef2ccc4e6cb0ab581b86f24c98
I20260926 20:46:47.485374 13578 ts_tablet_manager.cc:1939] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 90f1f7a896dd4c63be76af2920e1366a: Deleting consensus metadata
I20260926 20:46:47.487588 13028 catalog_manager.cc:4841] TS 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657): tablet 72b026ef2ccc4e6cb0ab581b86f24c98 (table test-table [id=5cf1e0353c0b47b8b0b18b4a78a43b56]) successfully deleted
I20260926 20:46:47.488530 13576 ts_tablet_manager.cc:1918] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:47.488790 13576 log.cc:1196] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/72b026ef2ccc4e6cb0ab581b86f24c98
I20260926 20:46:47.489056 13554 tablet_bootstrap.cc:492] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: No bootstrap required, opened a new log
I20260926 20:46:47.489276 13554 ts_tablet_manager.cc:1397] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260926 20:46:47.489367 13576 ts_tablet_manager.cc:1939] T 72b026ef2ccc4e6cb0ab581b86f24c98 P 18b20c16f9d242f9b1dd570d171f390b: Deleting consensus metadata
I20260926 20:46:47.490180 13554 raft_consensus.cc:348] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.490541 13554 raft_consensus.cc:374] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.490688 13554 raft_consensus.cc:729] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.491062 13554 consensus_queue.cc:260] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.491271 13028 catalog_manager.cc:4841] TS 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741): tablet 72b026ef2ccc4e6cb0ab581b86f24c98 (table test-table [id=5cf1e0353c0b47b8b0b18b4a78a43b56]) successfully deleted
I20260926 20:46:47.494732 13554 ts_tablet_manager.cc:1428] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 20:46:47.495165 13552 tablet_bootstrap.cc:492] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: No bootstrap required, opened a new log
I20260926 20:46:47.495332 13552 ts_tablet_manager.cc:1397] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.009s
I20260926 20:46:47.496344 13577 ts_tablet_manager.cc:1918] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:47.496249 13552 raft_consensus.cc:348] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.496472 13552 raft_consensus.cc:374] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.496575 13552 raft_consensus.cc:729] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.496701 13577 log.cc:1196] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/72b026ef2ccc4e6cb0ab581b86f24c98
I20260926 20:46:47.496865 13552 consensus_queue.cc:260] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.497603 13552 ts_tablet_manager.cc:1428] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 20:46:47.497997 13577 ts_tablet_manager.cc:1939] T 72b026ef2ccc4e6cb0ab581b86f24c98 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting consensus metadata
I20260926 20:46:47.499745 13028 catalog_manager.cc:4841] TS ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179): tablet 72b026ef2ccc4e6cb0ab581b86f24c98 (table test-table [id=5cf1e0353c0b47b8b0b18b4a78a43b56]) successfully deleted
I20260926 20:46:47.763255 13560 raft_consensus.cc:482] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:47.763530 13560 raft_consensus.cc:504] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.765054 13560 leader_election.cc:288] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657), 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:47.772930 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.766471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36659 (local address 127.5.78.3:34741)
0926 20:46:47.766649 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.766655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.767314 (+ 659us) server_negotiation.cc:407] Connection header received
0926 20:46:47.767504 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.767510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.767596 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.767696 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.768172 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.768865 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.770905 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.771873 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.771994 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.772092 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.772486 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.772626 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:46:47.773118 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.766292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:34741 (local address 127.5.78.2:36659)
0926 20:46:47.767171 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.767195 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.767346 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.767719 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.767724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.767746 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.768019 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.768026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.768989 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.768997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.770757 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.770768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.772120 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.772130 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.772307 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.772343 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.772451 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.772585 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":106,"thread_start_us":529,"threads_started":1}
I20260926 20:46:47.773936 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.765372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43657 (local address 127.5.78.2:44329)
0926 20:46:47.766128 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.766149 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.766333 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.767027 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.767030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.767051 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:47.767462 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.767475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.769041 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.769051 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.771273 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.771288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.773088 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.773094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.773262 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.773293 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.773448 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.773586 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":2,"thread_start_us":475,"threads_started":1}
I20260926 20:46:47.773994 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.765389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:44329 (local address 127.5.78.1:43657)
0926 20:46:47.765657 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.765667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.766226 (+ 559us) server_negotiation.cc:407] Connection header received
0926 20:46:47.766592 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.766601 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.766708 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.766795 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:47.767646 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.768868 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.771439 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.772660 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.772797 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.772889 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.773479 (+ 590us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.773631 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:46:47.773983 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f09d669769647a6b599e197535ad215" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b" is_pre_election: true
I20260926 20:46:47.774248 13478 raft_consensus.cc:2454] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 0.
I20260926 20:46:47.774740 13271 leader_election.cc:302] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b20c16f9d242f9b1dd570d171f390b, ec2d4e81fe364ff2ae8dd30154f97988; no voters:
I20260926 20:46:47.775172 13560 raft_consensus.cc:2781] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:47.775318 13560 raft_consensus.cc:482] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:47.775170 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f09d669769647a6b599e197535ad215" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1f7a896dd4c63be76af2920e1366a" is_pre_election: true
I20260926 20:46:47.775418 13560 raft_consensus.cc:3037] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.775611 13199 raft_consensus.cc:2454] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 0.
I20260926 20:46:47.778673 13560 raft_consensus.cc:504] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.779440 13560 leader_election.cc:288] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 election: Requested vote from peers 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657), 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:47.780047 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f09d669769647a6b599e197535ad215" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1f7a896dd4c63be76af2920e1366a"
I20260926 20:46:47.780192 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f09d669769647a6b599e197535ad215" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b"
I20260926 20:46:47.780303 13199 raft_consensus.cc:3037] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.780467 13478 raft_consensus.cc:3037] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.783792 13478 raft_consensus.cc:2454] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 1.
I20260926 20:46:47.784129 13199 raft_consensus.cc:2454] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 1.
I20260926 20:46:47.784260 13271 leader_election.cc:302] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b20c16f9d242f9b1dd570d171f390b, ec2d4e81fe364ff2ae8dd30154f97988; no voters:
I20260926 20:46:47.784615 13560 raft_consensus.cc:2781] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:47.785152 13560 raft_consensus.cc:686] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 LEADER]: Becoming Leader. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Running, Role: LEADER
I20260926 20:46:47.785643 13560 consensus_queue.cc:237] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.789670 13041 catalog_manager.cc:5495] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } health_report { overall_health: UNKNOWN } } }
Sep 26 20:46:47 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455605, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.62@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:47.864871 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.848960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.5.78.62:44175)
0926 20:46:47.849616 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.849622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.849890 (+ 268us) server_negotiation.cc:407] Connection header received
0926 20:46:47.850072 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.850076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.850134 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.850228 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:47.851681 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.852722 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.854661 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.855166 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.855288 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.859275 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:47.859280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:47.859283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:47.859314 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:47.861969 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:47.862797 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:47.862807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:47.862809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:47.862937 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:47.863309 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:47.863318 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:47.863320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:47.863584 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:47.863754 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:47.864157 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 20:46:47.864508 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":170,"server-negotiator.queue_time_us":485,"thread_start_us":292,"threads_started":1}
I20260926 20:46:47.864892 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.848899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44175 (local address 127.0.0.1:34978)
0926 20:46:47.849770 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.849796 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.849960 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.850294 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.850299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.851222 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:47.851506 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.851523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.852870 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.852876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.854477 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.854493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.855357 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:47.855364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:47.855531 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:47.856376 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:47.856399 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:47.859038 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:47.862121 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:47.862138 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:47.862146 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:47.862670 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:47.863046 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:47.863052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:47.863056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:47.863198 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:47.863729 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:47.863737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:47.863910 (+ 173us) client_negotiation.cc:769] Sending connection context
0926 20:46:47.864075 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 20:46:47.864410 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":1,"thread_start_us":631,"threads_started":1}
I20260926 20:46:47.870260 13041 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:34978:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 20:46:47.871402 13041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:47.878342 13178 tablet_service.cc:1467] Processing CreateTablet for tablet 3375a8a0122041a5b3228158584f57bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e4f43b75dfc42039ebec0389f8625d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:46:47.878393 13318 tablet_service.cc:1467] Processing CreateTablet for tablet 3375a8a0122041a5b3228158584f57bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e4f43b75dfc42039ebec0389f8625d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:46:47.878485 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 3375a8a0122041a5b3228158584f57bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e4f43b75dfc42039ebec0389f8625d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:46:47.878964 13318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3375a8a0122041a5b3228158584f57bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.878966 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3375a8a0122041a5b3228158584f57bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.878984 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3375a8a0122041a5b3228158584f57bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:47.886492 13552 tablet_bootstrap.cc:492] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: Bootstrap starting.
I20260926 20:46:47.886672 13554 tablet_bootstrap.cc:492] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: Bootstrap starting.
I20260926 20:46:47.886747 13553 tablet_bootstrap.cc:492] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: Bootstrap starting.
I20260926 20:46:47.888324 13554 tablet_bootstrap.cc:654] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.888329 13552 tablet_bootstrap.cc:654] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.888392 13553 tablet_bootstrap.cc:654] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:47.896786 13553 tablet_bootstrap.cc:492] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: No bootstrap required, opened a new log
I20260926 20:46:47.896786 13554 tablet_bootstrap.cc:492] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: No bootstrap required, opened a new log
I20260926 20:46:47.896991 13553 ts_tablet_manager.cc:1397] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260926 20:46:47.896976 13554 ts_tablet_manager.cc:1397] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260926 20:46:47.897742 13553 raft_consensus.cc:348] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.897742 13554 raft_consensus.cc:348] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.897982 13552 tablet_bootstrap.cc:492] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: No bootstrap required, opened a new log
I20260926 20:46:47.897981 13554 raft_consensus.cc:374] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.897995 13553 raft_consensus.cc:374] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.898059 13554 raft_consensus.cc:729] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.898066 13553 raft_consensus.cc:729] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b20c16f9d242f9b1dd570d171f390b, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.898147 13552 ts_tablet_manager.cc:1397] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260926 20:46:47.898295 13553 consensus_queue.cc:260] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.898299 13554 consensus_queue.cc:260] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.898855 13553 ts_tablet_manager.cc:1428] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:46:47.898777 13560 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:47.898885 13552 raft_consensus.cc:348] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.899057 13560 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:47.899107 13552 raft_consensus.cc:374] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:47.899220 13552 raft_consensus.cc:729] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Initialized, Role: FOLLOWER
I20260926 20:46:47.899322 13559 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:47.899462 13552 consensus_queue.cc:260] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.899639 13559 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:47.899863 13563 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:47.900133 13563 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:47.900204 13554 ts_tablet_manager.cc:1428] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:46:47.902899 13552 ts_tablet_manager.cc:1428] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 20:46:47.971577 13563 raft_consensus.cc:482] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:47.971832 13563 raft_consensus.cc:504] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.972499 13563 leader_election.cc:288] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179), 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:47.973073 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3375a8a0122041a5b3228158584f57bb" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" is_pre_election: true
I20260926 20:46:47.973088 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3375a8a0122041a5b3228158584f57bb" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b" is_pre_election: true
I20260926 20:46:47.973337 13344 raft_consensus.cc:2454] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 0.
I20260926 20:46:47.973352 13478 raft_consensus.cc:2454] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 0.
I20260926 20:46:47.973814 13131 leader_election.cc:302] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f1f7a896dd4c63be76af2920e1366a, ec2d4e81fe364ff2ae8dd30154f97988; no voters:
I20260926 20:46:47.974133 13563 raft_consensus.cc:2781] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:47.974242 13563 raft_consensus.cc:482] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:47.974352 13563 raft_consensus.cc:3037] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.976403 13563 raft_consensus.cc:504] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.977059 13563 leader_election.cc:288] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 election: Requested vote from peers ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179), 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:47.977753 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3375a8a0122041a5b3228158584f57bb" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2d4e81fe364ff2ae8dd30154f97988"
I20260926 20:46:47.977790 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3375a8a0122041a5b3228158584f57bb" candidate_uuid: "90f1f7a896dd4c63be76af2920e1366a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b"
I20260926 20:46:47.978029 13478 raft_consensus.cc:3037] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.978021 13344 raft_consensus.cc:3037] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:47.981033 13478 raft_consensus.cc:2454] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 1.
I20260926 20:46:47.981050 13344 raft_consensus.cc:2454] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f1f7a896dd4c63be76af2920e1366a in term 1.
I20260926 20:46:47.981501 13131 leader_election.cc:302] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f1f7a896dd4c63be76af2920e1366a, ec2d4e81fe364ff2ae8dd30154f97988; no voters:
I20260926 20:46:47.981813 13563 raft_consensus.cc:2781] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:47.982342 13563 raft_consensus.cc:686] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [term 1 LEADER]: Becoming Leader. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Running, Role: LEADER
I20260926 20:46:47.982695 13563 consensus_queue.cc:237] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:47.984467 13596 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: TxnStatusTablet state changed. Reason: New leader 90f1f7a896dd4c63be76af2920e1366a. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:47.984861 13596 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:46:47.985422 13598 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 20:46:47.985636 13598 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 20:46:47.986116 13042 catalog_manager.cc:5495] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1). New cstate: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } health_report { overall_health: UNKNOWN } } }
I20260926 20:46:48.003948 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.997416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.5.78.1:43657)
0926 20:46:47.997603 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:47.997609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:47.997892 (+ 283us) server_negotiation.cc:407] Connection header received
0926 20:46:47.998124 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:47.998131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:47.998237 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:47.998352 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:47.998750 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:47.999928 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.001827 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.002515 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.002645 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.003159 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.003291 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.003386 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.003544 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.003636 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:46:48.003975 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:47.997428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43657 (local address 127.0.0.1:58856)
0926 20:46:47.997789 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:47.997810 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:47.997945 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:47.998385 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:47.998390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:47.998433 (+ 43us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:47.998610 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:47.998620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.000082 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.000088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.001653 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.001672 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.002680 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.002686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.002888 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.002964 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.003403 (+ 439us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.003421 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.003499 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.003661 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260926 20:46:48.008474 13344 raft_consensus.cc:1261] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Refusing update from remote peer 90f1f7a896dd4c63be76af2920e1366a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.008482 13478 raft_consensus.cc:1261] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Refusing update from remote peer 90f1f7a896dd4c63be76af2920e1366a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.009107 13596 consensus_queue.cc:1035] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.009470 13563 consensus_queue.cc:1035] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.019778 13559 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: TxnStatusTablet state changed. Reason: New leader 90f1f7a896dd4c63be76af2920e1366a. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:48.020144 13559 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:48.022478 13601 mvcc.cc:205] Tried to move back new op lower bound from 7333706170396073984 to 7333706170299674624. Current Snapshot: MvccSnapshot[applied={T|T < 7333706170396073984}]
I20260926 20:46:48.022526 13560 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: TxnStatusTablet state changed. Reason: New leader 90f1f7a896dd4c63be76af2920e1366a. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:48.022785 13560 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:48.024410 13603 mvcc.cc:205] Tried to move back new op lower bound from 7333706170396073984 to 7333706170299674624. Current Snapshot: MvccSnapshot[applied={T|T < 7333706170396073984}]
I20260926 20:46:48.030957 13563 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:48.031450 13563 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:46:48.038125 13559 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:48.038393 13559 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 18b20c16f9d242f9b1dd570d171f390b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:48.039908 13560 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:48.040205 13560 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P ec2d4e81fe364ff2ae8dd30154f97988: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:46:48.046056 13596 tablet_replica.cc:439] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90f1f7a896dd4c63be76af2920e1366a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } } }
I20260926 20:46:48.046878 13596 tablet_replica.cc:442] T 3375a8a0122041a5b3228158584f57bb P 90f1f7a896dd4c63be76af2920e1366a: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:46:48.056422 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.047144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.5.78.2:36179)
0926 20:46:48.049853 (+ 2709us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.049860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.050061 (+ 201us) server_negotiation.cc:407] Connection header received
0926 20:46:48.050325 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.050329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.050464 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.050669 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.051163 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.052144 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.054107 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.054582 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.054674 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.055161 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.055721 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.055827 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.055989 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.056054 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2583,"thread_start_us":2452,"threads_started":1}
I20260926 20:46:48.073745 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.067189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43657 (local address 127.5.78.2:36993)
0926 20:46:48.067899 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:48.067915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:48.068035 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:48.068397 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:48.068401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:48.068410 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:48.068702 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.068710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.070113 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.070118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.071308 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.071321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.072435 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.072441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.072576 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.072725 (+ 149us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.073216 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.073244 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.073316 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.073452 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":397,"threads_started":1}
I20260926 20:46:48.073770 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.067223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36993 (local address 127.5.78.1:43657)
0926 20:46:48.067468 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.067479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.067993 (+ 514us) server_negotiation.cc:407] Connection header received
0926 20:46:48.068163 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.068169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.068264 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.068368 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.068832 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.069941 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.071463 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.072209 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.072336 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.072912 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.073069 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.073161 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.073362 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.073453 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:46:48.083127 13478 raft_consensus.cc:1261] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Refusing update from remote peer ec2d4e81fe364ff2ae8dd30154f97988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.083472 13199 raft_consensus.cc:1261] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Refusing update from remote peer ec2d4e81fe364ff2ae8dd30154f97988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.083962 13584 consensus_queue.cc:1035] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.084821 13560 consensus_queue.cc:1035] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.121410 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.113691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:50397 (local address 127.5.78.2:36179)
0926 20:46:48.113873 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.113879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.114608 (+ 729us) server_negotiation.cc:407] Connection header received
0926 20:46:48.114837 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.114843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.114926 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.115019 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.115758 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.116836 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.119024 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.119685 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.119821 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.120779 (+ 958us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.120902 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.120975 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.121117 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.121179 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:46:48.121711 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.113681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:36179 (local address 127.5.78.1:50397)
0926 20:46:48.114505 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:48.114527 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:48.114689 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:48.115177 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:48.115182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:48.115195 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:48.115577 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.115598 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.116969 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.116977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.118867 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.118883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.120300 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.120306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.120473 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.120560 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.121004 (+ 444us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.121017 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.121106 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.121270 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":431,"threads_started":1}
I20260926 20:46:48.161700 13042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:48.162083 13042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:48.166484 13042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Sending DeleteTablet for 3 replicas of tablet 5f09d669769647a6b599e197535ad215
I20260926 20:46:48.168134 13178 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f09d669769647a6b599e197535ad215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:58848
I20260926 20:46:48.169647 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f09d669769647a6b599e197535ad215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:60064
I20260926 20:46:48.170394 13318 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f09d669769647a6b599e197535ad215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:38076
I20260926 20:46:48.194134 13626 tablet_replica.cc:330] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: stopping tablet replica
I20260926 20:46:48.194468 13626 raft_consensus.cc:2229] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:48.193954 13627 tablet_replica.cc:330] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: stopping tablet replica
I20260926 20:46:48.194770 13627 raft_consensus.cc:2229] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:48.194878 13626 raft_consensus.cc:2258] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.196035 13627 raft_consensus.cc:2258] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.196190 13626 ts_tablet_manager.cc:1905] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.203619 13626 ts_tablet_manager.cc:1918] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:48.203874 13626 log.cc:1196] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/5f09d669769647a6b599e197535ad215
I20260926 20:46:48.204409 13627 ts_tablet_manager.cc:1905] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.204453 13626 ts_tablet_manager.cc:1939] T 5f09d669769647a6b599e197535ad215 P 18b20c16f9d242f9b1dd570d171f390b: Deleting consensus metadata
I20260926 20:46:48.205617 13028 catalog_manager.cc:4841] TS 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741): tablet 5f09d669769647a6b599e197535ad215 (table test-table [id=7b30c21959c7493f8f41933f90788901]) successfully deleted
I20260926 20:46:48.205809 13625 tablet_replica.cc:330] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: stopping tablet replica
I20260926 20:46:48.206017 13625 raft_consensus.cc:2229] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:48.206323 13625 raft_consensus.cc:2258] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.211338 13625 ts_tablet_manager.cc:1905] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.214730 13627 ts_tablet_manager.cc:1918] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:48.214980 13627 log.cc:1196] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/5f09d669769647a6b599e197535ad215
I20260926 20:46:48.215559 13627 ts_tablet_manager.cc:1939] T 5f09d669769647a6b599e197535ad215 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting consensus metadata
I20260926 20:46:48.216899 13028 catalog_manager.cc:4841] TS ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179): tablet 5f09d669769647a6b599e197535ad215 (table test-table [id=7b30c21959c7493f8f41933f90788901]) successfully deleted
I20260926 20:46:48.220276 13625 ts_tablet_manager.cc:1918] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:48.220543 13625 log.cc:1196] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/5f09d669769647a6b599e197535ad215
I20260926 20:46:48.221194 13625 ts_tablet_manager.cc:1939] T 5f09d669769647a6b599e197535ad215 P 90f1f7a896dd4c63be76af2920e1366a: Deleting consensus metadata
I20260926 20:46:48.222563 13028 catalog_manager.cc:4841] TS 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657): tablet 5f09d669769647a6b599e197535ad215 (table test-table [id=7b30c21959c7493f8f41933f90788901]) successfully deleted
Sep 26 20:46:48 dist-test-slave-z7j8 krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:46:48 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:48 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:46:48.280198 13592 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:48.266331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.5.78.62:44175)
0926 20:46:48.266577 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.266586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.267022 (+ 436us) server_negotiation.cc:407] Connection header received
0926 20:46:48.267228 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.267234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.267319 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.267435 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:48.269297 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.270493 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.272455 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.273134 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.273299 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.279865 (+ 6566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34994: BlockingRecv error: recv got EOF from 127.0.0.1:34994 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 20:46:48 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM
I20260926 20:46:48.302460 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.284148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.5.78.62:44175)
0926 20:46:48.284304 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.284310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.284763 (+ 453us) server_negotiation.cc:407] Connection header received
0926 20:46:48.284935 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.284942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.285034 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.285152 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:48.286638 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.287799 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.289579 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.290287 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.290455 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.295463 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:48.295470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:48.295476 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:48.295522 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:48.299258 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:48.300140 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:48.300147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:48.300150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:48.300340 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:48.300789 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:48.300794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:48.300797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:48.301143 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:48.301415 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.301778 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.302030 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 20:46:48.306977 13042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:48.307788 13042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:48.315585 13178 tablet_service.cc:1467] Processing CreateTablet for tablet df38b9e64090432889df070c0d4130e5 (DEFAULT_TABLE table=test-table [id=a7d2a19c3e294f689deff7113262c38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:48.316130 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df38b9e64090432889df070c0d4130e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:48.319624 13318 tablet_service.cc:1467] Processing CreateTablet for tablet df38b9e64090432889df070c0d4130e5 (DEFAULT_TABLE table=test-table [id=a7d2a19c3e294f689deff7113262c38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:48.320113 13318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df38b9e64090432889df070c0d4130e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:48.320667 13457 tablet_service.cc:1467] Processing CreateTablet for tablet df38b9e64090432889df070c0d4130e5 (DEFAULT_TABLE table=test-table [id=a7d2a19c3e294f689deff7113262c38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:48.321103 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df38b9e64090432889df070c0d4130e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:48.323539 13552 tablet_bootstrap.cc:492] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Bootstrap starting.
I20260926 20:46:48.325611 13552 tablet_bootstrap.cc:654] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:48.329505 13553 tablet_bootstrap.cc:492] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Bootstrap starting.
I20260926 20:46:48.331807 13553 tablet_bootstrap.cc:654] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:48.332376 13554 tablet_bootstrap.cc:492] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Bootstrap starting.
I20260926 20:46:48.334352 13554 tablet_bootstrap.cc:654] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:48.352305 13554 tablet_bootstrap.cc:492] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: No bootstrap required, opened a new log
I20260926 20:46:48.352556 13553 tablet_bootstrap.cc:492] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: No bootstrap required, opened a new log
I20260926 20:46:48.352619 13554 ts_tablet_manager.cc:1397] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.014s
I20260926 20:46:48.352762 13553 ts_tablet_manager.cc:1397] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.001s
I20260926 20:46:48.353600 13553 raft_consensus.cc:348] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.353904 13553 raft_consensus.cc:374] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:48.354027 13553 raft_consensus.cc:729] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b20c16f9d242f9b1dd570d171f390b, State: Initialized, Role: FOLLOWER
I20260926 20:46:48.353732 13554 raft_consensus.cc:348] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.354286 13554 raft_consensus.cc:374] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:48.354336 13553 consensus_queue.cc:260] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.354493 13554 raft_consensus.cc:729] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Initialized, Role: FOLLOWER
I20260926 20:46:48.354897 13554 consensus_queue.cc:260] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.357443 13553 ts_tablet_manager.cc:1428] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 20:46:48.358085 13554 ts_tablet_manager.cc:1428] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 20:46:48.360747 13552 tablet_bootstrap.cc:492] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: No bootstrap required, opened a new log
I20260926 20:46:48.360922 13552 ts_tablet_manager.cc:1397] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Time spent bootstrapping tablet: real 0.038s user 0.005s sys 0.005s
I20260926 20:46:48.361790 13552 raft_consensus.cc:348] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.362205 13552 raft_consensus.cc:374] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:48.362350 13552 raft_consensus.cc:729] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Initialized, Role: FOLLOWER
I20260926 20:46:48.362731 13552 consensus_queue.cc:260] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.363368 13552 ts_tablet_manager.cc:1428] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:46:48.404289 13621 raft_consensus.cc:482] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:48.404594 13621 raft_consensus.cc:504] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.405395 13621 leader_election.cc:288] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657), 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:48.406100 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df38b9e64090432889df070c0d4130e5" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1f7a896dd4c63be76af2920e1366a" is_pre_election: true
I20260926 20:46:48.406232 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df38b9e64090432889df070c0d4130e5" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b" is_pre_election: true
I20260926 20:46:48.406386 13199 raft_consensus.cc:2454] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 0.
I20260926 20:46:48.406554 13478 raft_consensus.cc:2454] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 0.
I20260926 20:46:48.406957 13271 leader_election.cc:302] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f1f7a896dd4c63be76af2920e1366a, ec2d4e81fe364ff2ae8dd30154f97988; no voters:
I20260926 20:46:48.407342 13621 raft_consensus.cc:2781] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:48.407496 13621 raft_consensus.cc:482] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:48.407616 13621 raft_consensus.cc:3037] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:48.411158 13621 raft_consensus.cc:504] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.412580 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df38b9e64090432889df070c0d4130e5" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1f7a896dd4c63be76af2920e1366a"
I20260926 20:46:48.412842 13199 raft_consensus.cc:3037] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:48.413050 13621 leader_election.cc:288] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 election: Requested vote from peers 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657), 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741)
I20260926 20:46:48.413846 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df38b9e64090432889df070c0d4130e5" candidate_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b20c16f9d242f9b1dd570d171f390b"
I20260926 20:46:48.414077 13478 raft_consensus.cc:3037] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:48.416288 13199 raft_consensus.cc:2454] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 1.
I20260926 20:46:48.416930 13271 leader_election.cc:302] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f1f7a896dd4c63be76af2920e1366a, ec2d4e81fe364ff2ae8dd30154f97988; no voters:
I20260926 20:46:48.417018 13478 raft_consensus.cc:2454] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec2d4e81fe364ff2ae8dd30154f97988 in term 1.
I20260926 20:46:48.417461 13621 raft_consensus.cc:2781] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:48.417696 13621 raft_consensus.cc:686] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 LEADER]: Becoming Leader. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Running, Role: LEADER
I20260926 20:46:48.418134 13621 consensus_queue.cc:237] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.421530 13042 catalog_manager.cc:5495] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } health_report { overall_health: UNKNOWN } } }
I20260926 20:46:48.461052 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.453271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.5.78.2:36179)
0926 20:46:48.453486 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.453495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.453523 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:46:48.453609 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.453613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.453687 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.453845 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.454350 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.455614 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.457858 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.458716 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.458883 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.459650 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.459805 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.459926 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.460542 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.460648 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 20:46:48.465517 13478 raft_consensus.cc:1261] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Refusing update from remote peer ec2d4e81fe364ff2ae8dd30154f97988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.465548 13199 raft_consensus.cc:1261] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Refusing update from remote peer ec2d4e81fe364ff2ae8dd30154f97988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.466313 13618 consensus_queue.cc:1035] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.466646 13621 consensus_queue.cc:1035] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.492020 13041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:48.492300 13041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:48.500099 13041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Sending DeleteTablet for 3 replicas of tablet df38b9e64090432889df070c0d4130e5
I20260926 20:46:48.501780 13318 tablet_service.cc:1514] Processing DeleteTablet for tablet df38b9e64090432889df070c0d4130e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:38076
I20260926 20:46:48.502027 13178 tablet_service.cc:1514] Processing DeleteTablet for tablet df38b9e64090432889df070c0d4130e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:58848
I20260926 20:46:48.502147 13627 tablet_replica.cc:330] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: stopping tablet replica
I20260926 20:46:48.502152 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet df38b9e64090432889df070c0d4130e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:60064
I20260926 20:46:48.502314 13625 tablet_replica.cc:330] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: stopping tablet replica
I20260926 20:46:48.502368 13627 raft_consensus.cc:2229] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:48.502544 13625 raft_consensus.cc:2229] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:48.502707 13627 raft_consensus.cc:2258] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.502779 13625 raft_consensus.cc:2258] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.503052 13626 tablet_replica.cc:330] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: stopping tablet replica
I20260926 20:46:48.503247 13626 raft_consensus.cc:2229] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:48.503454 13626 raft_consensus.cc:2258] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.503854 13627 ts_tablet_manager.cc:1905] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.507848 13626 ts_tablet_manager.cc:1905] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.510836 13039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:46:48.511658 13039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:46:48.511963 13627 ts_tablet_manager.cc:1918] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:48.512171 13627 log.cc:1196] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/df38b9e64090432889df070c0d4130e5
I20260926 20:46:48.512738 13627 ts_tablet_manager.cc:1939] T df38b9e64090432889df070c0d4130e5 P ec2d4e81fe364ff2ae8dd30154f97988: Deleting consensus metadata
I20260926 20:46:48.515596 13028 catalog_manager.cc:4841] TS ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179): tablet df38b9e64090432889df070c0d4130e5 (table test-table [id=a7d2a19c3e294f689deff7113262c38e]) successfully deleted
I20260926 20:46:48.522799 13178 tablet_service.cc:1467] Processing CreateTablet for tablet a89001ef40204b9385a3dec5c695c44e (DEFAULT_TABLE table=test-table [id=fbaa1c4a82c7486ebeff5d2456a3d919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:48.523336 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89001ef40204b9385a3dec5c695c44e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:48.523331 13318 tablet_service.cc:1467] Processing CreateTablet for tablet a89001ef40204b9385a3dec5c695c44e (DEFAULT_TABLE table=test-table [id=fbaa1c4a82c7486ebeff5d2456a3d919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:48.523767 13318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89001ef40204b9385a3dec5c695c44e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:48.527397 13457 tablet_service.cc:1467] Processing CreateTablet for tablet a89001ef40204b9385a3dec5c695c44e (DEFAULT_TABLE table=test-table [id=fbaa1c4a82c7486ebeff5d2456a3d919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:46:48.527848 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89001ef40204b9385a3dec5c695c44e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:48.528124 13626 ts_tablet_manager.cc:1918] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:48.528265 13626 log.cc:1196] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/df38b9e64090432889df070c0d4130e5
I20260926 20:46:48.528769 13626 ts_tablet_manager.cc:1939] T df38b9e64090432889df070c0d4130e5 P 18b20c16f9d242f9b1dd570d171f390b: Deleting consensus metadata
I20260926 20:46:48.530823 13028 catalog_manager.cc:4841] TS 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741): tablet df38b9e64090432889df070c0d4130e5 (table test-table [id=a7d2a19c3e294f689deff7113262c38e]) successfully deleted
I20260926 20:46:48.530987 13554 tablet_bootstrap.cc:492] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Bootstrap starting.
I20260926 20:46:48.531392 13625 ts_tablet_manager.cc:1905] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.533012 13554 tablet_bootstrap.cc:654] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:48.535609 13554 tablet_bootstrap.cc:492] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: No bootstrap required, opened a new log
I20260926 20:46:48.535795 13554 ts_tablet_manager.cc:1397] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260926 20:46:48.536715 13554 raft_consensus.cc:348] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.537077 13554 raft_consensus.cc:374] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:48.537223 13554 raft_consensus.cc:729] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2d4e81fe364ff2ae8dd30154f97988, State: Initialized, Role: FOLLOWER
I20260926 20:46:48.537583 13554 consensus_queue.cc:260] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.538177 13554 ts_tablet_manager.cc:1428] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 20:46:48.545073 13625 ts_tablet_manager.cc:1918] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:46:48.545264 13625 log.cc:1196] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/df38b9e64090432889df070c0d4130e5
I20260926 20:46:48.545768 13625 ts_tablet_manager.cc:1939] T df38b9e64090432889df070c0d4130e5 P 90f1f7a896dd4c63be76af2920e1366a: Deleting consensus metadata
I20260926 20:46:48.547058 13028 catalog_manager.cc:4841] TS 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657): tablet df38b9e64090432889df070c0d4130e5 (table test-table [id=a7d2a19c3e294f689deff7113262c38e]) successfully deleted
I20260926 20:46:48.551812 13553 tablet_bootstrap.cc:492] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Bootstrap starting.
I20260926 20:46:48.552330 13552 tablet_bootstrap.cc:492] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Bootstrap starting.
I20260926 20:46:48.553561 13553 tablet_bootstrap.cc:654] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:48.554085 13552 tablet_bootstrap.cc:654] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:48.563555 13553 tablet_bootstrap.cc:492] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: No bootstrap required, opened a new log
I20260926 20:46:48.563825 13553 ts_tablet_manager.cc:1397] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260926 20:46:48.564725 13553 raft_consensus.cc:348] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.565070 13553 raft_consensus.cc:374] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:48.565196 13553 raft_consensus.cc:729] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b20c16f9d242f9b1dd570d171f390b, State: Initialized, Role: FOLLOWER
I20260926 20:46:48.565423 13552 tablet_bootstrap.cc:492] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: No bootstrap required, opened a new log
I20260926 20:46:48.565627 13552 ts_tablet_manager.cc:1397] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.009s
I20260926 20:46:48.565526 13553 consensus_queue.cc:260] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.566558 13552 raft_consensus.cc:348] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.567035 13552 raft_consensus.cc:374] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:48.567188 13552 raft_consensus.cc:729] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f1f7a896dd4c63be76af2920e1366a, State: Initialized, Role: FOLLOWER
I20260926 20:46:48.567554 13552 consensus_queue.cc:260] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.568652 13553 ts_tablet_manager.cc:1428] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260926 20:46:48.568832 13552 ts_tablet_manager.cc:1428] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:46:48.751734 13559 raft_consensus.cc:482] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:46:48.752036 13559 raft_consensus.cc:504] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.753618 13559 leader_election.cc:288] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657), ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179)
I20260926 20:46:48.759804 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.754017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:47263 (local address 127.5.78.1:43657)
0926 20:46:48.754532 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.754537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.754705 (+ 168us) server_negotiation.cc:407] Connection header received
0926 20:46:48.754887 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.754895 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.754981 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.755135 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:48.755606 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.756278 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.757981 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.758841 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.758990 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.759098 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.759364 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.759504 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":250,"threads_started":1}
I20260926 20:46:48.759918 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.753925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43657 (local address 127.5.78.3:47263)
0926 20:46:48.754593 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:48.754615 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:48.754752 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:48.755157 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:48.755161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:48.755186 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:48.755410 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.755419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.756426 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.756431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.757815 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.757827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.759023 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.759029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.759198 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.759223 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.759328 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.759463 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":24,"thread_start_us":352,"threads_started":1}
I20260926 20:46:48.760461 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.754863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:56239 (local address 127.5.78.2:36179)
0926 20:46:48.755100 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.755107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.755359 (+ 252us) server_negotiation.cc:407] Connection header received
0926 20:46:48.755582 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.755586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.755639 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.755729 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:46:48.756220 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.756890 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.758623 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.759465 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.759611 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.759720 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.760078 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.760201 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 20:46:48.760800 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89001ef40204b9385a3dec5c695c44e" candidate_uuid: "18b20c16f9d242f9b1dd570d171f390b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1f7a896dd4c63be76af2920e1366a" is_pre_election: true
I20260926 20:46:48.761042 13199 raft_consensus.cc:2454] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18b20c16f9d242f9b1dd570d171f390b in term 0.
I20260926 20:46:48.761175 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.754706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:36179 (local address 127.5.78.3:56239)
0926 20:46:48.755252 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:48.755269 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:48.755416 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:48.755753 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:48.755757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:48.755781 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:46:48.756082 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.756090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.757034 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.757042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.758487 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.758498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.759665 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.759671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.759862 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.759912 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.760723 (+ 811us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.760828 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":1,"thread_start_us":397,"threads_started":1}
I20260926 20:46:48.761521 13411 leader_election.cc:302] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b20c16f9d242f9b1dd570d171f390b, 90f1f7a896dd4c63be76af2920e1366a; no voters:
I20260926 20:46:48.761869 13559 raft_consensus.cc:2781] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:46:48.762001 13559 raft_consensus.cc:482] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:46:48.762120 13559 raft_consensus.cc:3037] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:48.762267 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89001ef40204b9385a3dec5c695c44e" candidate_uuid: "18b20c16f9d242f9b1dd570d171f390b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" is_pre_election: true
I20260926 20:46:48.762506 13344 raft_consensus.cc:2454] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18b20c16f9d242f9b1dd570d171f390b in term 0.
I20260926 20:46:48.764811 13559 raft_consensus.cc:504] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.765414 13559 leader_election.cc:288] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [CANDIDATE]: Term 1 election: Requested vote from peers 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657), ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179)
I20260926 20:46:48.765897 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89001ef40204b9385a3dec5c695c44e" candidate_uuid: "18b20c16f9d242f9b1dd570d171f390b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1f7a896dd4c63be76af2920e1366a"
I20260926 20:46:48.765936 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89001ef40204b9385a3dec5c695c44e" candidate_uuid: "18b20c16f9d242f9b1dd570d171f390b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2d4e81fe364ff2ae8dd30154f97988"
I20260926 20:46:48.766104 13199 raft_consensus.cc:3037] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:48.766121 13344 raft_consensus.cc:3037] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:48.768110 13199 raft_consensus.cc:2454] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18b20c16f9d242f9b1dd570d171f390b in term 1.
I20260926 20:46:48.768110 13344 raft_consensus.cc:2454] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18b20c16f9d242f9b1dd570d171f390b in term 1.
I20260926 20:46:48.768554 13411 leader_election.cc:302] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b20c16f9d242f9b1dd570d171f390b, 90f1f7a896dd4c63be76af2920e1366a; no voters:
I20260926 20:46:48.768885 13559 raft_consensus.cc:2781] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:48.769356 13559 raft_consensus.cc:686] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 1 LEADER]: Becoming Leader. State: Replica: 18b20c16f9d242f9b1dd570d171f390b, State: Running, Role: LEADER
I20260926 20:46:48.769812 13559 consensus_queue.cc:237] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } }
I20260926 20:46:48.773239 13042 catalog_manager.cc:5495] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b reported cstate change: term changed from 0 to 1, leader changed from <none> to 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3). New cstate: current_term: 1 leader_uuid: "18b20c16f9d242f9b1dd570d171f390b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18b20c16f9d242f9b1dd570d171f390b" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 34741 } health_report { overall_health: HEALTHY } } }
I20260926 20:46:48.802109 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.796240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.5.78.3:34741)
0926 20:46:48.796791 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.796796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.796815 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:46:48.796896 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.796900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.796976 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.797126 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.797461 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.798229 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.799592 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.800069 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.800191 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.800741 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.801377 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.801497 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.801652 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.801722 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":319,"threads_started":1}
I20260926 20:46:48.818511 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.813027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:43619 (local address 127.5.78.1:43657)
0926 20:46:48.813233 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.813237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.813930 (+ 693us) server_negotiation.cc:407] Connection header received
0926 20:46:48.814080 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.814084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.814150 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.814237 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.814815 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.815531 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.816896 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.817288 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.817389 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.817803 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.817905 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.817975 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.818130 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.818198 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:46:48.818563 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.813017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:43657 (local address 127.5.78.3:43619)
0926 20:46:48.813800 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:48.813816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:48.813949 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:48.814266 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:48.814269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:48.814279 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:48.814626 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.814635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.815686 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.815690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.816763 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.816775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.817444 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.817448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.817570 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.817648 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.817986 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.818000 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.818060 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.818171 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":420,"threads_started":1}
I20260926 20:46:48.826723 13344 raft_consensus.cc:1261] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Refusing update from remote peer 18b20c16f9d242f9b1dd570d171f390b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.826711 13199 raft_consensus.cc:1261] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Refusing update from remote peer 18b20c16f9d242f9b1dd570d171f390b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:46:48.827379 13653 consensus_queue.cc:1035] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f1f7a896dd4c63be76af2920e1366a" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.827740 13559 consensus_queue.cc:1035] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec2d4e81fe364ff2ae8dd30154f97988" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:46:48.853997 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.848974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:34105 (local address 127.5.78.3:34741)
0926 20:46:48.849181 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.849187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.849378 (+ 191us) server_negotiation.cc:407] Connection header received
0926 20:46:48.849495 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.849498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.849556 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.849627 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:46:48.849957 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.850846 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.852688 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.853079 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.853181 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.853482 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.853568 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.853630 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.853723 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.853776 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:46:48.854055 13666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.848787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:34741 (local address 127.5.78.1:34105)
0926 20:46:48.849287 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:48.849303 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:48.849412 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:48.849634 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:48.849637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:48.849646 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:46:48.849845 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.849852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.850976 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.850981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.852586 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:48.852598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.853217 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.853222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:48.853337 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.853373 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:46:48.853646 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:46:48.853653 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:46:48.853713 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 20:46:48.853809 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":244,"threads_started":1}
I20260926 20:46:48.894176 13042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:48.894562 13042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:46:48.898169 13042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a12ebe8eab8f4e1ca46663a4b7d2650f: Sending DeleteTablet for 3 replicas of tablet a89001ef40204b9385a3dec5c695c44e
I20260926 20:46:48.899335 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet a89001ef40204b9385a3dec5c695c44e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:60064
I20260926 20:46:48.899421 13318 tablet_service.cc:1514] Processing DeleteTablet for tablet a89001ef40204b9385a3dec5c695c44e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:38076
I20260926 20:46:48.899421 13178 tablet_service.cc:1514] Processing DeleteTablet for tablet a89001ef40204b9385a3dec5c695c44e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:46:48 UTC) from {username='oryx', principal='oryx/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:58848
I20260926 20:46:48.899704 13626 tablet_replica.cc:330] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: stopping tablet replica
I20260926 20:46:48.899725 13625 tablet_replica.cc:330] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: stopping tablet replica
I20260926 20:46:48.899891 13626 raft_consensus.cc:2229] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:46:48.899890 13625 raft_consensus.cc:2229] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:48.900113 13625 raft_consensus.cc:2258] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.900439 13626 raft_consensus.cc:2258] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.900740 13627 tablet_replica.cc:330] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: stopping tablet replica
I20260926 20:46:48.900907 13627 raft_consensus.cc:2229] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:46:48.901124 13627 raft_consensus.cc:2258] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:46:48.901762 13625 ts_tablet_manager.cc:1905] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.901857 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13113
I20260926 20:46:48.902141 13627 ts_tablet_manager.cc:1905] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.907526 13627 ts_tablet_manager.cc:1918] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:48.907691 13627 log.cc:1196] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/a89001ef40204b9385a3dec5c695c44e
I20260926 20:46:48.908147 13627 ts_tablet_manager.cc:1939] T a89001ef40204b9385a3dec5c695c44e P ec2d4e81fe364ff2ae8dd30154f97988: Deleting consensus metadata
I20260926 20:46:48.908274 13626 ts_tablet_manager.cc:1905] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:46:48.911661 13028 catalog_manager.cc:4841] TS ec2d4e81fe364ff2ae8dd30154f97988 (127.5.78.2:36179): tablet a89001ef40204b9385a3dec5c695c44e (table test-table [id=fbaa1c4a82c7486ebeff5d2456a3d919]) successfully deleted
I20260926 20:46:48.917032 13626 ts_tablet_manager.cc:1918] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:48.917197 13626 log.cc:1196] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/a89001ef40204b9385a3dec5c695c44e
I20260926 20:46:48.917645 13626 ts_tablet_manager.cc:1939] T a89001ef40204b9385a3dec5c695c44e P 18b20c16f9d242f9b1dd570d171f390b: Deleting consensus metadata
Sep 26 20:46:48 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.1@KRBTEST.COM
I20260926 20:46:48.919056 13028 catalog_manager.cc:4841] TS 18b20c16f9d242f9b1dd570d171f390b (127.5.78.3:34741): tablet a89001ef40204b9385a3dec5c695c44e (table test-table [id=fbaa1c4a82c7486ebeff5d2456a3d919]) successfully deleted
I20260926 20:46:48.921762 13625 ts_tablet_manager.cc:1918] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:46:48.921921 13625 log.cc:1196] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/a89001ef40204b9385a3dec5c695c44e
I20260926 20:46:48.922437 13625 ts_tablet_manager.cc:1939] T a89001ef40204b9385a3dec5c695c44e P 90f1f7a896dd4c63be76af2920e1366a: Deleting consensus metadata
I20260926 20:46:48.923659 13028 catalog_manager.cc:4841] TS 90f1f7a896dd4c63be76af2920e1366a (127.5.78.1:43657): tablet a89001ef40204b9385a3dec5c695c44e (table test-table [id=fbaa1c4a82c7486ebeff5d2456a3d919]) successfully deleted
I20260926 20:46:48.927732 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:48.906925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.5.78.1:43657)
0926 20:46:48.907311 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:48.907317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:48.907526 (+ 209us) server_negotiation.cc:407] Connection header received
0926 20:46:48.908068 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:48.908073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:48.908172 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:48.908254 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:48.910110 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.911312 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.913688 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:48.914373 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:48.914505 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:48.920140 (+ 5635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:48.920159 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:48.920179 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:48.920221 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:48.924470 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:48.925424 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:48.925434 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:48.925438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:48.925604 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:48.926041 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:48.926046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:48.926049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:48.926539 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:48.926842 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:48.927157 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 20:46:48.927369 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260926 20:46:48.928311 13244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:49.085726 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13113
I20260926 20:46:49.112565 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13253
Sep 26 20:46:49 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.2@KRBTEST.COM
I20260926 20:46:49.127869 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:49.113373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.5.78.2:36179)
0926 20:46:49.113537 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:49.113542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:49.113606 (+ 64us) server_negotiation.cc:407] Connection header received
0926 20:46:49.113831 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:49.113837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:49.113912 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:49.114022 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:49.115280 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:49.115970 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:49.117189 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:49.117606 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:49.117746 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:49.121808 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:49.121824 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:49.121839 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:49.121879 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:49.124874 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:49.125555 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:49.125562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:49.125564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:49.125700 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:49.126067 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:49.126073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:49.126076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:49.126705 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:49.127015 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:49.127313 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 20:46:49.127541 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:46:49.128513 13384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:49.253472 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13253
I20260926 20:46:49.276696 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13393
Sep 26 20:46:49 dist-test-slave-z7j8 krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.78.3@KRBTEST.COM
I20260926 20:46:49.295182 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:49.277718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.5.78.3:34741)
0926 20:46:49.277952 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:49.277959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:49.277980 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:46:49.278065 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:49.278071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:49.278156 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:49.278285 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:49.279663 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:49.280853 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:49.282132 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:49.282949 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:49.283098 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:49.287819 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:49.287846 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:49.287868 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:49.287925 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:49.292038 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:49.292808 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:49.292824 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:49.292833 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:49.293014 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:49.293380 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:49.293385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:49.293388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:49.293907 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:49.294232 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:49.294593 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 20:46:49.294841 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:46:49.295989 13523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:49.430871 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13393
I20260926 20:46:49.452972 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13010
I20260926 20:46:49.454242 13082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:49.567745 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13010
2026-09-26T20:46:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6159 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:49 dist-test-slave-z7j8 krb5kdc[13676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:49 dist-test-slave-z7j8 krb5kdc[13676](info): set up 2 sockets
Sep 26 20:46:49 dist-test-slave-z7j8 krb5kdc[13676](info): commencing operation
krb5kdc: starting...
W20260926 20:46:50.679435 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:50 dist-test-slave-z7j8 krb5kdc[13676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:50.922204 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44339
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:51.260231 13689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:51.260545 13689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:51.260703 13689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:51.268127 13689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:51.268244 13689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:51.268337 13689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:51.268425 13689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:51.268517 13689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:51.278685 13689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:51.280036 13689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:51.281437 13689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:51.293516 13695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:51.293450 13694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:51.294121 13697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:51.294620 13689 server_base.cc:1034] running on GCE node
I20260926 20:46:51.295457 13689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:51.297044 13689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:51.298269 13689 hybrid_clock.cc:648] HybridClock initialized: now 1790455611298206 us; error 61 us; skew 500 ppm
I20260926 20:46:51.298668 13689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T20:46:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1791 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:46:51 dist-test-slave-z7j8 krb5kdc[13703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:46:51 dist-test-slave-z7j8 krb5kdc[13703](info): set up 2 sockets
Sep 26 20:46:51 dist-test-slave-z7j8 krb5kdc[13703](info): commencing operation
krb5kdc: starting...
W20260926 20:46:52.467424 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:46:52 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:46:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:46:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:46:52.694876 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:32769
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:32769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:53.027128 13716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:53.027500 13716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:53.027673 13716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:53.035032 13716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:53.035223 13716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:53.035315 13716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:53.035400 13716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:53.035468 13716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:53.045454 13716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:32769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:32769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:53.046927 13716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:53.048470 13716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:53.060400 13722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:53.060443 13721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:53.060359 13724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:53.061336 13716 server_base.cc:1034] running on GCE node
I20260926 20:46:53.062022 13716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:53.066570 13716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:53.067829 13716 hybrid_clock.cc:648] HybridClock initialized: now 1790455613067746 us; error 58 us; skew 500 ppm
I20260926 20:46:53.068300 13716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:53 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455613, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:53.071923 13716 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:53.073428 13716 webserver.cc:460] Webserver started at http://127.5.78.62:44351/ using document root <none> and password file <none>
I20260926 20:46:53.074151 13716 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:53.074298 13716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:53.074679 13716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:53.077690 13716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "c74eb7a51bd945df8a87440554da349f"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "c808ec309ec16df86893325cda100c63"
server_key_iv: "95a98cf96df6b10bf61cb90267ba2fa0"
server_key_version: "encryptionkey@0"
I20260926 20:46:53.078768 13716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "c74eb7a51bd945df8a87440554da349f"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "c808ec309ec16df86893325cda100c63"
server_key_iv: "95a98cf96df6b10bf61cb90267ba2fa0"
server_key_version: "encryptionkey@0"
I20260926 20:46:53.084419 13716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:46:53.088261 13731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:53.089795 13716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:53.090015 13716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c74eb7a51bd945df8a87440554da349f"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "c808ec309ec16df86893325cda100c63"
server_key_iv: "95a98cf96df6b10bf61cb90267ba2fa0"
server_key_version: "encryptionkey@0"
I20260926 20:46:53.090233 13716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:53.105422 13716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:53.109603 13716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:53.109853 13716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:53.128597 13716 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:32769
I20260926 20:46:53.128628 13782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:32769 every 8 connection(s)
I20260926 20:46:53.129992 13716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:53.133716 13783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:46:53.137959 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13716
I20260926 20:46:53.138170 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:46:53.138976 5432 external_mini_cluster.cc:1428] Setting key e222c61ab4eb47d242b91876f03a2649
I20260926 20:46:53.145495 13783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Bootstrap starting.
I20260926 20:46:53.149365 13783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Neither blocks nor log segments found. Creating new log.
I20260926 20:46:53.150660 13783 log.cc:824] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:53.153463 13783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: No bootstrap required, opened a new log
Sep 26 20:46:53 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:53.162678 13783 raft_consensus.cc:348] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:53.163061 13783 raft_consensus.cc:374] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:46:53.163070 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:53.141050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.5.78.62:32769)
0926 20:46:53.141804 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:53.141818 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:53.141864 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:46:53.143981 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:53.144007 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:53.144444 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:53.144797 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:53.146707 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:53.148139 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:53.149978 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:53.150841 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:53.150982 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:53.155452 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:53.155463 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:53.155478 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:53.155518 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:53.158845 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:53.159689 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:53.159696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:53.159702 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:53.159885 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:53.160272 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:53.160275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:53.160277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:53.160819 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:53.161026 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:53.161503 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 20:46:53.161900 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":337,"threads_started":1}
I20260926 20:46:53.163203 13783 raft_consensus.cc:729] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c74eb7a51bd945df8a87440554da349f, State: Initialized, Role: FOLLOWER
I20260926 20:46:53.163818 13783 consensus_queue.cc:260] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:53.164070 13783 raft_consensus.cc:388] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:53.164227 13783 raft_consensus.cc:482] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:53.164402 13783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:46:53.166882 13783 raft_consensus.cc:504] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:53.167356 13783 leader_election.cc:302] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c74eb7a51bd945df8a87440554da349f; no voters:
I20260926 20:46:53.167903 13783 leader_election.cc:288] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:46:53.168038 13788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:46:53.168875 13788 raft_consensus.cc:686] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 LEADER]: Becoming Leader. State: Replica: c74eb7a51bd945df8a87440554da349f, State: Running, Role: LEADER
I20260926 20:46:53.169457 13788 consensus_queue.cc:237] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:53.170037 13783 sys_catalog.cc:564] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:53.172262 13790 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c74eb7a51bd945df8a87440554da349f. Latest consensus state: current_term: 1 leader_uuid: "c74eb7a51bd945df8a87440554da349f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } } }
I20260926 20:46:53.172673 13790 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:53.173040 13789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c74eb7a51bd945df8a87440554da349f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } } }
I20260926 20:46:53.173409 13789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:53.175981 13795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:53.181851 13795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:53.192044 13795 catalog_manager.cc:1347] Generated new cluster ID: ec4fdf4ef53f4b66981df464160174f1
I20260926 20:46:53.192171 13795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:53.205386 13795 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:46:53.207396 13795 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:53.220610 13795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Generated new TSK 0
I20260926 20:46:53.221788 13795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:46:53.280061 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32769
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:53.614697 13809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:53.615006 13809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:53.615192 13809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:53.622750 13809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:53.622879 13809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:53.623096 13809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:53.633054 13809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:53.634426 13809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:53.635797 13809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:53.649243 13814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:53.649297 13815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:53.649263 13817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:53.650563 13809 server_base.cc:1034] running on GCE node
I20260926 20:46:53.651306 13809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:53.652307 13809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:53.653559 13809 hybrid_clock.cc:648] HybridClock initialized: now 1790455613653515 us; error 33 us; skew 500 ppm
I20260926 20:46:53.653924 13809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:53 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455613, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:53.657356 13809 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:46:53.659014 13809 webserver.cc:460] Webserver started at http://127.5.78.1:42741/ using document root <none> and password file <none>
I20260926 20:46:53.659740 13809 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:53.659888 13809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:53.660203 13809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:53.663249 13809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "64ca5547af4e4322959a0f67e733c247"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "1e3cc1ec24e0dda08d6bdce321dc92a0"
server_key_iv: "9d5a8957348bfa9e6b44f95b2010fe92"
server_key_version: "encryptionkey@0"
I20260926 20:46:53.664069 13809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "64ca5547af4e4322959a0f67e733c247"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "1e3cc1ec24e0dda08d6bdce321dc92a0"
server_key_iv: "9d5a8957348bfa9e6b44f95b2010fe92"
server_key_version: "encryptionkey@0"
I20260926 20:46:53.669726 13809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:46:53.673656 13824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:53.675565 13809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:53.675740 13809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "64ca5547af4e4322959a0f67e733c247"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "1e3cc1ec24e0dda08d6bdce321dc92a0"
server_key_iv: "9d5a8957348bfa9e6b44f95b2010fe92"
server_key_version: "encryptionkey@0"
I20260926 20:46:53.675959 13809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:53.698987 13809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:53.703560 13809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:53.703866 13809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:53.704828 13809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:53.706164 13809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:53.706277 13809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:53.706458 13809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:53.706543 13809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:46:53.738693 13809 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:44489
I20260926 20:46:53.738740 13936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:44489 every 8 connection(s)
I20260926 20:46:53.739960 13809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:53.741560 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13809
I20260926 20:46:53.741761 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:46:53.742506 5432 external_mini_cluster.cc:1428] Setting key 3416ebc60ecaf78aa741f6c90bf6b88a
Sep 26 20:46:53 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455613, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:53.761998 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:53.742673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:55905 (local address 127.5.78.62:32769)
0926 20:46:53.743198 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:53.743204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:53.745122 (+ 1918us) server_negotiation.cc:407] Connection header received
0926 20:46:53.747049 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:53.747057 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:53.747146 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:53.747266 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:53.748798 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:53.750006 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:53.751723 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:53.752136 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:53.752283 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:53.756270 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:53.756275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:53.756279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:53.756311 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:53.758857 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:53.759581 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:53.759586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:53.759589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:53.759749 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:53.760283 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:53.760286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:53.760289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:53.760523 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:53.760628 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:53.761585 (+ 957us) server_negotiation.cc:299] Negotiation successful
0926 20:46:53.761745 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":234,"threads_started":1}
I20260926 20:46:53.763643 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:53.743258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:55905)
0926 20:46:53.744948 (+ 1690us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:53.744996 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:53.746767 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:53.747494 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:53.747503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:53.748029 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:53.748603 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:53.748620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:53.750183 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:53.750188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:53.751524 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:53.751536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:53.752377 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:53.752387 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:53.752635 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:53.753413 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:53.753439 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:53.756022 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:53.759001 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:53.759010 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:53.759028 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:53.759446 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:53.759893 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:53.759900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:53.759906 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:53.760165 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:53.760656 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:53.760668 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:53.761164 (+ 496us) client_negotiation.cc:769] Sending connection context
0926 20:46:53.761512 (+ 348us) client_negotiation.cc:240] Negotiation successful
0926 20:46:53.762113 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1416,"thread_start_us":1216,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:46:53.765452 13937 heartbeater.cc:347] Connected to a master server at 127.5.78.62:32769
I20260926 20:46:53.765784 13937 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:53.766570 13937 heartbeater.cc:511] Master 127.5.78.62:32769 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:46:53.768668 13748 ts_manager.cc:194] Registered new tserver with Master: 64ca5547af4e4322959a0f67e733c247 (127.5.78.1:44489)
I20260926 20:46:53.770138 13748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:55905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:46:53.793725 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32769
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:54.118778 13943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:54.119071 13943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:54.119243 13943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:54.126507 13943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:54.126631 13943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:54.126854 13943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:54.136839 13943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:54.138088 13943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:54.139493 13943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:54.152560 13949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:54.152586 13948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:54.153199 13951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:54.153411 13943 server_base.cc:1034] running on GCE node
I20260926 20:46:54.154134 13943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:54.155306 13943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:54.156545 13943 hybrid_clock.cc:648] HybridClock initialized: now 1790455614156466 us; error 71 us; skew 500 ppm
I20260926 20:46:54.156919 13943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:54 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455614, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:54.160388 13943 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:46:54.161902 13943 webserver.cc:460] Webserver started at http://127.5.78.2:41725/ using document root <none> and password file <none>
I20260926 20:46:54.162727 13943 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:54.162870 13943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:54.163180 13943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:54.166141 13943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "087349ad88274c7bb1e7c444ff0aeb81"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "3278372fd775e518db3366ececdaa2aa"
server_key_iv: "561f89f056c387f9c0c9b1219054c3a7"
server_key_version: "encryptionkey@0"
I20260926 20:46:54.167018 13943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "087349ad88274c7bb1e7c444ff0aeb81"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "3278372fd775e518db3366ececdaa2aa"
server_key_iv: "561f89f056c387f9c0c9b1219054c3a7"
server_key_version: "encryptionkey@0"
I20260926 20:46:54.172554 13943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:46:54.176229 13958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:54.177565 13943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:46:54.177731 13943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "087349ad88274c7bb1e7c444ff0aeb81"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "3278372fd775e518db3366ececdaa2aa"
server_key_iv: "561f89f056c387f9c0c9b1219054c3a7"
server_key_version: "encryptionkey@0"
I20260926 20:46:54.177924 13943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:54.203194 13943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:54.207458 13943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:54.207741 13943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:54.208680 13943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:54.210035 13943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:54.210142 13943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:54.210287 13943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:54.210358 13943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:54.244051 13943 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:40565
I20260926 20:46:54.244187 14070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:40565 every 8 connection(s)
I20260926 20:46:54.245311 13943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:54.245638 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 13943
I20260926 20:46:54.245819 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:46:54.246440 5432 external_mini_cluster.cc:1428] Setting key 18521d05fd5fcf32f1194cc6c6f08880
Sep 26 20:46:54 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455614, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:54.269204 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:54.247899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:33075 (local address 127.5.78.62:32769)
0926 20:46:54.248078 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:54.248084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:54.249427 (+ 1343us) server_negotiation.cc:407] Connection header received
0926 20:46:54.252330 (+ 2903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:54.252334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:54.252406 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:54.252494 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:54.254618 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.255293 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.257440 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.257878 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.258017 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:54.261620 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:54.261630 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:54.261638 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:54.261683 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:54.265614 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:54.266526 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:54.266533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:54.266537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:54.266701 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:54.267197 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:54.267205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:54.267209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:54.267621 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:54.267792 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:54.268537 (+ 745us) server_negotiation.cc:299] Negotiation successful
0926 20:46:54.268790 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:46:54.270049 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:54.248417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:33075)
0926 20:46:54.249241 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:54.249300 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:54.252020 (+ 2720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:54.252775 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:54.252788 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:54.253592 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:54.254449 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:54.254479 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.255455 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.255463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:54.257298 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:54.257310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.258106 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.258112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:54.258349 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:54.259095 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:54.259117 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:54.261355 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:54.265799 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:54.265808 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:54.265824 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:54.266328 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:54.266830 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:54.266834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:54.266836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:54.267058 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:54.267766 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:54.267772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:54.268125 (+ 353us) client_negotiation.cc:769] Sending connection context
0926 20:46:54.268382 (+ 257us) client_negotiation.cc:240] Negotiation successful
0926 20:46:54.268800 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":295,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:46:54.271711 14071 heartbeater.cc:347] Connected to a master server at 127.5.78.62:32769
I20260926 20:46:54.272027 14071 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:54.272917 14071 heartbeater.cc:511] Master 127.5.78.62:32769 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:46:54.274545 13748 ts_manager.cc:194] Registered new tserver with Master: 087349ad88274c7bb1e7c444ff0aeb81 (127.5.78.2:40565)
I20260926 20:46:54.275764 13748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:33075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:46:54.302393 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32769
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:54.636310 14076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:54.636619 14076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:54.636790 14076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:54.644167 14076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:54.644295 14076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:54.644483 14076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:54.654703 14076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:54.656098 14076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:54.657444 14076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:54.671693 14081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:54.672096 14084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:54.671666 14082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:54.672420 14076 server_base.cc:1034] running on GCE node
I20260926 20:46:54.673101 14076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:54.674182 14076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:54.675446 14076 hybrid_clock.cc:648] HybridClock initialized: now 1790455614675388 us; error 57 us; skew 500 ppm
I20260926 20:46:54.675840 14076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:54 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455614, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:54.679210 14076 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:46:54.680873 14076 webserver.cc:460] Webserver started at http://127.5.78.3:39921/ using document root <none> and password file <none>
I20260926 20:46:54.681704 14076 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:54.681845 14076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:54.682179 14076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:46:54.685247 14076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "b0086a1b68f54ef2a0da6ea7d46c5975"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "6d241b1f4cd1bd269d5e632be973c3aa"
server_key_iv: "9c577a81a75aebd99e341a668372c78f"
server_key_version: "encryptionkey@0"
I20260926 20:46:54.686088 14076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "b0086a1b68f54ef2a0da6ea7d46c5975"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "6d241b1f4cd1bd269d5e632be973c3aa"
server_key_iv: "9c577a81a75aebd99e341a668372c78f"
server_key_version: "encryptionkey@0"
I20260926 20:46:54.691632 14076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:46:54.695534 14091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:54.697075 14076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:46:54.697279 14076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "b0086a1b68f54ef2a0da6ea7d46c5975"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "6d241b1f4cd1bd269d5e632be973c3aa"
server_key_iv: "9c577a81a75aebd99e341a668372c78f"
server_key_version: "encryptionkey@0"
I20260926 20:46:54.697492 14076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:54.714149 14076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:54.718269 14076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:54.718595 14076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:54.719583 14076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:54.721028 14076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:54.721102 14076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:54.721227 14076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:54.721292 14076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:54.755324 14076 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38467
I20260926 20:46:54.755414 14203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38467 every 8 connection(s)
I20260926 20:46:54.756615 14076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:54.765170 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14076
I20260926 20:46:54.765367 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:46:54.765800 5432 external_mini_cluster.cc:1428] Setting key 470e313566fb970cb7744901c359e980
Sep 26 20:46:54 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455614, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:46:54.773038 13937 heartbeater.cc:503] Master 127.5.78.62:32769 was elected leader, sending a full tablet report...
I20260926 20:46:54.776998 13939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:54.759221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:44721 (local address 127.5.78.62:32769)
0926 20:46:54.759405 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:54.759409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:54.760612 (+ 1203us) server_negotiation.cc:407] Connection header received
0926 20:46:54.762348 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:54.762352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:54.762469 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:54.762567 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:54.764051 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.764779 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.766660 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.767073 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.767210 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:54.770739 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:54.770744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:54.770747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:54.770784 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:54.773534 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:54.774494 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:54.774503 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:54.774509 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:54.774667 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:54.775138 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:54.775145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:54.775149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:54.775442 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:54.775599 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:54.776382 (+ 783us) server_negotiation.cc:299] Negotiation successful
0926 20:46:54.776628 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:46:54.778003 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:54.759713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:44721)
0926 20:46:54.760446 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:54.760484 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:54.762092 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:54.762811 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:54.762819 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:54.763334 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:54.763886 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:54.763903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.764960 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.764964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:54.766473 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:54.766490 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.767301 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.767308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:54.767558 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:54.768230 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:54.768258 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:54.770534 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:46:54.773665 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:54.773673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:54.773707 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:54.774317 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:54.774787 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:46:54.774792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:46:54.774796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:46:54.775020 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:46:54.775577 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:46:54.775586 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:46:54.776004 (+ 418us) client_negotiation.cc:769] Sending connection context
0926 20:46:54.776274 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 20:46:54.776888 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":290,"threads_started":1}
I20260926 20:46:54.779518 14204 heartbeater.cc:347] Connected to a master server at 127.5.78.62:32769
I20260926 20:46:54.779914 14204 heartbeater.cc:464] Registering TS with master...
I20260926 20:46:54.781032 14204 heartbeater.cc:511] Master 127.5.78.62:32769 requested a full tablet report, sending...
I20260926 20:46:54.782516 13748 ts_manager.cc:194] Registered new tserver with Master: b0086a1b68f54ef2a0da6ea7d46c5975 (127.5.78.3:38467)
I20260926 20:46:54.783329 13748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:44721
I20260926 20:46:54.793874 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab.
Entry for principal oryx/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab.
WARNING: no policy specified for oryx/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab.
Entry for principal oryx/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab.
WARNING: no policy specified for oryx/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab.
Entry for principal oryx/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab.
I20260926 20:46:54.938230 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13809
Sep 26 20:46:54 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:46:54.956393 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:54.939730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.5.78.1:44489)
0926 20:46:54.940368 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:54.940375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:54.940407 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:46:54.940472 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:54.940478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:54.940707 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:54.940865 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:54.942175 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.943303 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.944617 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:54.945220 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:54.945337 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:54.950024 (+ 4687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:54.950039 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:54.950054 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:54.950097 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:54.953314 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:54.954121 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:54.954130 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:54.954132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:54.954287 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:54.954747 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:54.954753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:54.954758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:54.955119 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:54.955440 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:54.955748 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 20:46:54.956024 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":316,"threads_started":1}
I20260926 20:46:54.957733 13932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:55.040742 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13809
I20260926 20:46:55.056957 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13943
Sep 26 20:46:55 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:46:55.079898 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:55.057918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.5.78.2:40565)
0926 20:46:55.058456 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:55.058467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:55.058502 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:46:55.058563 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:55.058570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:55.058784 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:55.058915 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:55.060190 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:55.061192 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:55.062674 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:55.064060 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:55.064825 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:55.071908 (+ 7083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:55.071928 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:55.071949 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:55.072016 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:55.076420 (+ 4404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:55.077574 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:55.077587 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:55.077592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:55.077732 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:55.078137 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:55.078142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:55.078146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:55.078608 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:55.078935 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:55.079236 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 20:46:55.079524 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":218,"threads_started":1}
I20260926 20:46:55.081617 14066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:55.166675 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13943
I20260926 20:46:55.182955 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14076
Sep 26 20:46:55 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:46:55.200788 14218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:55.183988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.5.78.3:38467)
0926 20:46:55.184616 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:55.184623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:55.184653 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:46:55.184711 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:55.184717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:55.184927 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:55.185128 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:55.186489 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:55.187552 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:55.188827 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:55.189401 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:55.189546 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:55.194274 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:55.194290 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:55.194334 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:55.194381 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:55.198025 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:55.198935 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:55.198943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:55.198946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:55.199101 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:55.199441 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:55.199444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:55.199446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:55.199796 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:55.200038 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:55.200315 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 20:46:55.200502 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":332,"threads_started":1}
I20260926 20:46:55.202107 14199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:55.284750 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14076
I20260926 20:46:55.300575 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 13716
I20260926 20:46:55.301914 13778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:46:55.403003 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 13716
I20260926 20:46:55.423566 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:32769
--webserver_interface=127.5.78.62
--webserver_port=44351
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:32769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:46:55.752306 14221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:55.752599 14221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:55.752772 14221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:55.759895 14221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:46:55.760010 14221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:55.760089 14221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:55.760187 14221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:46:55.760279 14221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:46:55.770066 14221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:32769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:32769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=44351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:55.771514 14221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:55.772893 14221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:55.784559 14226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:55.784585 14227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:55.784618 14229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:55.785387 14221 server_base.cc:1034] running on GCE node
I20260926 20:46:55.786139 14221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:55.787676 14221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:55.788990 14221 hybrid_clock.cc:648] HybridClock initialized: now 1790455615788920 us; error 58 us; skew 500 ppm
I20260926 20:46:55.789374 14221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:55 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455615, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:55.792601 14221 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:46:55.794229 14221 webserver.cc:460] Webserver started at http://127.5.78.62:44351/ using document root <none> and password file <none>
I20260926 20:46:55.794975 14221 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:55.795097 14221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:55.800555 14221 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:46:55.803491 14236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:55.804957 14221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:55.805130 14221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c74eb7a51bd945df8a87440554da349f"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "c808ec309ec16df86893325cda100c63"
server_key_iv: "95a98cf96df6b10bf61cb90267ba2fa0"
server_key_version: "encryptionkey@0"
I20260926 20:46:55.806107 14221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:55.832247 14221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:55.837121 14221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:55.837430 14221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:55.856581 14221 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:32769
I20260926 20:46:55.856631 14287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:32769 every 8 connection(s)
I20260926 20:46:55.858083 14221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:46:55.865151 14288 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:46:55.866118 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14221
I20260926 20:46:55.867100 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:44489
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:32769
--webserver_port=42741
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:46:55.870453 14288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Bootstrap starting.
I20260926 20:46:55.891201 14288 log.cc:824] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:46:55.900516 14288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:46:55.901126 14288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Bootstrap complete.
I20260926 20:46:55.910511 14288 raft_consensus.cc:348] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:55.911377 14288 raft_consensus.cc:729] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c74eb7a51bd945df8a87440554da349f, State: Initialized, Role: FOLLOWER
I20260926 20:46:55.912030 14288 consensus_queue.cc:260] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:55.912263 14288 raft_consensus.cc:388] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:46:55.912374 14288 raft_consensus.cc:482] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:46:55.912562 14288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:46:55.916457 14288 raft_consensus.cc:504] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:55.917001 14288 leader_election.cc:302] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c74eb7a51bd945df8a87440554da349f; no voters:
I20260926 20:46:55.917659 14288 leader_election.cc:288] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:46:55.917809 14292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:46:55.919193 14292 raft_consensus.cc:686] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [term 2 LEADER]: Becoming Leader. State: Replica: c74eb7a51bd945df8a87440554da349f, State: Running, Role: LEADER
I20260926 20:46:55.919798 14292 consensus_queue.cc:237] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } }
I20260926 20:46:55.919961 14288 sys_catalog.cc:564] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:46:55.922454 14294 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c74eb7a51bd945df8a87440554da349f. Latest consensus state: current_term: 2 leader_uuid: "c74eb7a51bd945df8a87440554da349f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } } }
I20260926 20:46:55.922827 14294 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:55.922585 14293 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c74eb7a51bd945df8a87440554da349f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74eb7a51bd945df8a87440554da349f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32769 } } }
I20260926 20:46:55.923034 14293 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f [sys.catalog]: This master's current role is: LEADER
I20260926 20:46:55.938581 14298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:46:55.944326 14298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:46:55.945717 14298 catalog_manager.cc:1259] Loaded cluster ID: ec4fdf4ef53f4b66981df464160174f1
I20260926 20:46:55.945827 14298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:46:55.949759 14298 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:46:55.951221 14298 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c74eb7a51bd945df8a87440554da349f: Loaded TSK: 0
I20260926 20:46:55.952732 14298 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:46:56.216580 14290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:56.216918 14290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:56.217093 14290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:56.224714 14290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:56.224885 14290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:56.225063 14290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:46:56.235183 14290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.1.keytab
--principal=oryx/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:44489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=42741
--tserver_master_addrs=127.5.78.62:32769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:56.236698 14290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:56.238065 14290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:56.251298 14314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:56.251505 14315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:56.251758 14317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:56.252398 14290 server_base.cc:1034] running on GCE node
I20260926 20:46:56.253150 14290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:56.254281 14290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:56.255530 14290 hybrid_clock.cc:648] HybridClock initialized: now 1790455616255425 us; error 102 us; skew 500 ppm
I20260926 20:46:56.255889 14290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455616, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:56.263695 14290 init.cc:377] Logged in from keytab as oryx/127.5.78.1@KRBTEST.COM (short username oryx)
I20260926 20:46:56.265571 14290 webserver.cc:460] Webserver started at http://127.5.78.1:42741/ using document root <none> and password file <none>
I20260926 20:46:56.266615 14290 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:56.266770 14290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:56.273864 14290 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.001s
I20260926 20:46:56.277369 14324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:56.279102 14290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:46:56.279280 14290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "64ca5547af4e4322959a0f67e733c247"
format_stamp: "Formatted at 2026-09-26 20:46:53 on dist-test-slave-z7j8"
server_key: "1e3cc1ec24e0dda08d6bdce321dc92a0"
server_key_iv: "9d5a8957348bfa9e6b44f95b2010fe92"
server_key_version: "encryptionkey@0"
I20260926 20:46:56.280608 14290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:56.303094 14290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:56.307124 14290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:56.307358 14290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:56.308252 14290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:56.309489 14290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:56.309583 14290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:56.309722 14290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:56.309778 14290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:56.342767 14290 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:44489
I20260926 20:46:56.342875 14436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:44489 every 8 connection(s)
I20260926 20:46:56.344218 14290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:46:56.352869 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14290
I20260926 20:46:56.353540 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:40565
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:32769
--webserver_port=41725
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:56.363994 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:56.347441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:48235)
0926 20:46:56.348215 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:56.348251 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:56.349862 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:56.351372 (+ 1510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:56.351380 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:56.351904 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:56.352665 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.352692 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.354328 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.354337 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.356601 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.356619 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.357637 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.357646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.357970 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.358734 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:56.358769 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:56.363029 (+ 4260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":360,"threads_started":1}
W20260926 20:46:56.364758 14437 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:32769 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.78.62:32769: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
W20260926 20:46:56.364830 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:56.347302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:48235 (local address 127.5.78.62:32769)
0926 20:46:56.348043 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:56.348061 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:56.348385 (+ 324us) server_negotiation.cc:407] Connection header received
0926 20:46:56.350463 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:56.350490 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:56.350850 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:56.351167 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:56.352907 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.354151 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.356748 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.357394 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.357530 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.363552 (+ 6022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:48235: BlockingRecv error: recv got EOF from 127.5.78.1:48235 (error 108)
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":240,"threads_started":1}
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:46:56.375699 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:56.365579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:54881 (local address 127.5.78.62:32769)
0926 20:46:56.365738 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:56.365742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:56.365757 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:46:56.365873 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:56.365876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:56.365930 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:56.366006 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:56.367044 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.367794 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.369859 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.370239 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.370331 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.375518 (+ 5187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:54881: BlockingRecv error: recv got EOF from 127.5.78.1:54881 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 20:46:56.375734 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:56.365374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:54881)
0926 20:46:56.365611 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:56.365634 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:56.365754 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:56.366078 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:56.366086 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:56.366642 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:56.366877 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.366892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.367962 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.367970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.369696 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.369716 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.370403 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.370445 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.370682 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.371328 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:56.371359 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:56.375308 (+ 3949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:56.385076 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:56.376592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:44037)
0926 20:46:56.376755 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:56.376775 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:56.376880 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:56.377195 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:56.377199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:56.377518 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:56.377661 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.377669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.378727 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.378733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.379748 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.379762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.380922 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.380926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.381047 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.381658 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:56.381675 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:56.384720 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 20:46:56.385330 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:56.376695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:44037 (local address 127.5.78.62:32769)
0926 20:46:56.376855 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:56.376860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:56.376885 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:46:56.376985 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:56.376990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:56.377066 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:56.377169 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:56.377800 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.378591 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.379916 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.380763 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.380937 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.384977 (+ 4040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:44037: BlockingRecv error: recv got EOF from 127.5.78.1:44037 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 20:46:56.385715 14437 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:46:56.695462 14441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:56.695806 14441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:56.695982 14441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:56.703904 14441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:56.704072 14441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:56.704281 14441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:46:56.714246 14441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.2.keytab
--principal=oryx/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:40565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=41725
--tserver_master_addrs=127.5.78.62:32769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:56.715648 14441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:56.716975 14441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:56.730370 14449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:56.730294 14447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:56.730341 14446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:56.731293 14441 server_base.cc:1034] running on GCE node
I20260926 20:46:56.731896 14441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:56.732858 14441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:56.734086 14441 hybrid_clock.cc:648] HybridClock initialized: now 1790455616734027 us; error 51 us; skew 500 ppm
I20260926 20:46:56.734463 14441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455616, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:56.737437 14441 init.cc:377] Logged in from keytab as oryx/127.5.78.2@KRBTEST.COM (short username oryx)
I20260926 20:46:56.738824 14441 webserver.cc:460] Webserver started at http://127.5.78.2:41725/ using document root <none> and password file <none>
I20260926 20:46:56.739500 14441 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:56.739651 14441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:56.745004 14441 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 20:46:56.748193 14456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:56.749857 14441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:46:56.750093 14441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "087349ad88274c7bb1e7c444ff0aeb81"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "3278372fd775e518db3366ececdaa2aa"
server_key_iv: "561f89f056c387f9c0c9b1219054c3a7"
server_key_version: "encryptionkey@0"
I20260926 20:46:56.751247 14441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:56.767426 14441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:56.771415 14441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:56.771695 14441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:56.772635 14441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:56.773921 14441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:56.774044 14441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:56.774183 14441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:56.774274 14441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:56.808301 14441 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:40565
I20260926 20:46:56.808391 14568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:40565 every 8 connection(s)
I20260926 20:46:56.809834 14441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:46:56.812841 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14441
I20260926 20:46:56.813468 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:38467
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:32769
--webserver_port=39921
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:46:56.828603 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:56.812666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:59199 (local address 127.5.78.62:32769)
0926 20:46:56.812885 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:56.812893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:56.814323 (+ 1430us) server_negotiation.cc:407] Connection header received
0926 20:46:56.816239 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:56.816247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:56.816338 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:56.816460 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:56.818033 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.819238 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.821847 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.822278 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.822379 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.828340 (+ 5961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:59199: BlockingRecv error: recv got EOF from 127.5.78.2:59199 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 20:46:56.829370 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:56.813281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:59199)
0926 20:46:56.814123 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:56.814179 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:56.815947 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:56.816710 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:56.816718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:56.817230 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:56.817815 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.817833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.819415 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.819424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.821679 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.821697 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.822531 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.822541 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.822893 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.823713 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:56.823752 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:56.828152 (+ 4400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":372,"threads_started":1}
W20260926 20:46:56.830120 14569 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:32769 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.78.62:32769: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:56.838943 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:56.830750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:36813)
0926 20:46:56.830923 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:56.830943 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:56.831056 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:56.831365 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:56.831370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:56.831704 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:56.831836 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.831843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.832838 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.832843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.833736 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.833747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.834499 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.834506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.834680 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.835358 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:56.835375 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:56.838634 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 20:46:56.839447 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:56.830888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36813 (local address 127.5.78.62:32769)
0926 20:46:56.831050 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:56.831055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:56.831075 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:46:56.831162 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:56.831167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:56.831237 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:56.831343 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:56.831979 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.832702 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.833882 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.834302 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.834468 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.838840 (+ 4372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:36813: BlockingRecv error: recv got EOF from 127.5.78.2:36813 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:56 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:56.848999 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:56.839727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:38483)
0926 20:46:56.839989 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:56.840008 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:56.840112 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:56.840407 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:56.840412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:56.840814 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:56.841010 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.841021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.842221 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.842226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.843842 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:56.843862 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.844593 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.844600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:56.844778 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.845409 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:56.845432 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:56.848779 (+ 3347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139}
W20260926 20:46:56.849381 14569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:46:56.849444 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:56.839950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:38483 (local address 127.5.78.62:32769)
0926 20:46:56.840124 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:56.840128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:56.840146 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:46:56.840203 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:56.840206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:56.840264 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:56.840348 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:56.841178 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.842069 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.844025 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:56.844439 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:56.844551 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:56.849036 (+ 4485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:38483: BlockingRecv error: recv got EOF from 127.5.78.2:38483 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 20:46:57.158298 14572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:46:57.158648 14572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:46:57.158823 14572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:46:57.166112 14572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:46:57.166211 14572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:46:57.166399 14572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:46:57.176084 14572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/krb5kdc/oryx_127.5.78.3.keytab
--principal=oryx/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:38467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=39921
--tserver_master_addrs=127.5.78.62:32769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:46:57.177378 14572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:46:57.178664 14572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:46:57.191480 14577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:57.191521 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:46:57.191568 14578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:46:57.197243 14572 server_base.cc:1034] running on GCE node
I20260926 20:46:57.197953 14572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:46:57.199043 14572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:46:57.200278 14572 hybrid_clock.cc:648] HybridClock initialized: now 1790455617200235 us; error 44 us; skew 500 ppm
I20260926 20:46:57.200610 14572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455617, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:46:57.204866 14572 init.cc:377] Logged in from keytab as oryx/127.5.78.3@KRBTEST.COM (short username oryx)
I20260926 20:46:57.206467 14572 webserver.cc:460] Webserver started at http://127.5.78.3:39921/ using document root <none> and password file <none>
I20260926 20:46:57.207201 14572 fs_manager.cc:359] Metadata directory not provided
I20260926 20:46:57.207324 14572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:46:57.213006 14572 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:46:57.216144 14587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:57.217599 14572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 20:46:57.217746 14572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "b0086a1b68f54ef2a0da6ea7d46c5975"
format_stamp: "Formatted at 2026-09-26 20:46:54 on dist-test-slave-z7j8"
server_key: "6d241b1f4cd1bd269d5e632be973c3aa"
server_key_iv: "9c577a81a75aebd99e341a668372c78f"
server_key_version: "encryptionkey@0"
I20260926 20:46:57.218822 14572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:46:57.267619 14572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:46:57.272142 14572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:46:57.272374 14572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:46:57.273308 14572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:46:57.274750 14572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:46:57.274838 14572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:57.274909 14572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:46:57.274969 14572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:46:57.308103 14572 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38467
I20260926 20:46:57.308168 14699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38467 every 8 connection(s)
I20260926 20:46:57.309803 14572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:46:57.319170 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14572
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:46:57.327826 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:57.312586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:42753 (local address 127.5.78.62:32769)
0926 20:46:57.312778 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:57.312782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:57.314112 (+ 1330us) server_negotiation.cc:407] Connection header received
0926 20:46:57.316097 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:57.316102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:57.316193 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:57.316308 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:57.317865 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.318648 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.320697 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.321455 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.321658 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.327528 (+ 5870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:42753: BlockingRecv error: recv got EOF from 127.5.78.3:42753 (error 108)
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":80}
I20260926 20:46:57.328591 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:57.313134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:42753)
0926 20:46:57.313931 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:57.313973 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:57.315804 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:57.316556 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:57.316572 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:57.317107 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:57.317684 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.317702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.318810 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.318814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.320503 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.320518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.321699 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.321707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.322070 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.322886 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:57.322923 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:57.327384 (+ 4461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":311,"threads_started":1}
W20260926 20:46:57.329471 14700 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:32769 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.78.62:32769: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:57.337090 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:57.320395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52836 (local address 127.5.78.62:32769)
0926 20:46:57.321007 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:57.321014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:57.321035 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:46:57.321175 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:57.321181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:57.321262 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:57.321378 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:57.323506 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.324324 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.326549 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.326971 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.327072 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.330172 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:46:57.330184 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:46:57.330205 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:46:57.330258 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:46:57.333479 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:57.334223 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:57.334240 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:57.334249 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:57.334502 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:46:57.334873 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:46:57.334881 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:46:57.334884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:46:57.335384 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:46:57.335664 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:46:57.336234 (+ 570us) server_negotiation.cc:299] Negotiation successful
0926 20:46:57.336648 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":491,"thread_start_us":365,"threads_started":1}
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:57.338383 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:57.330107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:33705)
0926 20:46:57.330298 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:57.330317 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:57.330450 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:57.330806 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:57.330810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:57.331266 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:57.331421 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.331429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.332437 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.332444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.333351 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.333362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.334078 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.334084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.334263 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.334852 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:57.334870 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:57.338114 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260926 20:46:57.338572 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:57.330229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:33705 (local address 127.5.78.62:32769)
0926 20:46:57.330392 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:57.330397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:57.330413 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:46:57.330601 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:57.330611 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:57.330675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:57.330783 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:57.331568 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.332303 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.333496 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.333923 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.334049 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.338324 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:33705: BlockingRecv error: recv got EOF from 127.5.78.3:33705 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:57.347121 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:57.339116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:50745)
0926 20:46:57.339289 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:57.339309 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:57.339433 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:57.339849 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:57.339852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:57.340153 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:57.340348 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.340357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.341454 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.341458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.342501 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.342520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.343181 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.343187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.343363 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.343939 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:57.343956 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:57.346810 (+ 2854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 20:46:57.347334 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:57.339206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:50745 (local address 127.5.78.62:32769)
0926 20:46:57.339418 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:57.339425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:57.339442 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:46:57.339546 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:57.339553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:57.339620 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:57.339710 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:57.340491 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.341337 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.342646 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.343036 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.343168 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.347022 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:50745: BlockingRecv error: recv got EOF from 127.5.78.3:50745 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260926 20:46:57.347445 14700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:57.395174 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:57.386536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:39861)
0926 20:46:57.387074 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:57.387091 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:57.387211 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:57.387528 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:57.387531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:57.387971 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:57.388158 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.388166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.389338 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.389344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.390739 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.390753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.391451 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.391455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.391568 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.392199 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:57.392216 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:57.394956 (+ 2740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":293,"threads_started":1}
W20260926 20:46:57.395583 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:57.386561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:39861 (local address 127.5.78.62:32769)
0926 20:46:57.386733 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:57.386737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:57.387190 (+ 453us) server_negotiation.cc:407] Connection header received
0926 20:46:57.387327 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:57.387332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:57.387398 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:57.387493 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:57.388286 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.389172 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.390873 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.391301 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.391396 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.395193 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:39861: BlockingRecv error: recv got EOF from 127.5.78.1:39861 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:57 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:57.858830 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:57.850078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:54667)
0926 20:46:57.850602 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:57.850618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:57.850780 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:57.851120 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:57.851126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:57.851608 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:57.851801 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.851810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.852810 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.852817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.853779 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:57.853791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.854607 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.854614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:57.854771 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.855424 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:57.855442 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:57.858529 (+ 3087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":277,"threads_started":1}
W20260926 20:46:57.858981 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:57.850153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:54667 (local address 127.5.78.62:32769)
0926 20:46:57.850352 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:57.850364 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:57.850732 (+ 368us) server_negotiation.cc:407] Connection header received
0926 20:46:57.850907 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:57.850913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:57.850984 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:57.851088 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:57.851953 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.852652 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.853939 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:57.854354 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:57.854540 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:57.858742 (+ 4202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:54667: BlockingRecv error: recv got EOF from 127.5.78.2:54667 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 20:46:58 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:58 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:58.357008 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:58.348001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:33585)
0926 20:46:58.348554 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:58.348571 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:58.348688 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:58.349084 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:58.349088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:58.349563 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:58.349758 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:58.349768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.351058 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.351062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:58.352039 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:58.352052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.352760 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.352764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:58.352886 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:58.353504 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:58.353522 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:58.356764 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":320,"threads_started":1}
W20260926 20:46:58.357259 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:58.348175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:33585 (local address 127.5.78.62:32769)
0926 20:46:58.348357 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:58.348361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:58.348675 (+ 314us) server_negotiation.cc:407] Connection header received
0926 20:46:58.348792 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:58.348812 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:58.348881 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:58.348974 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:58.349919 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.350907 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.352182 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.352602 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.352704 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:58.356965 (+ 4261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:33585: BlockingRecv error: recv got EOF from 127.5.78.3:33585 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 20:46:58 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:58 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:58.404343 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:58.396098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:33993)
0926 20:46:58.396620 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:58.396637 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:58.396785 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:58.397110 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:58.397115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:58.397559 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:58.397697 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:58.397705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.398786 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.398794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:58.399705 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:58.399716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.400401 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.400406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:58.400582 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:58.401205 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:58.401224 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:58.403991 (+ 2767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":278,"threads_started":1}
W20260926 20:46:58.404428 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:58.396180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:33993 (local address 127.5.78.62:32769)
0926 20:46:58.396372 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:58.396379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:58.396720 (+ 341us) server_negotiation.cc:407] Connection header received
0926 20:46:58.396912 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:58.396917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:58.396984 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:58.397082 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:58.397848 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.398656 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.399851 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.400244 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.400387 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:58.404194 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:33993: BlockingRecv error: recv got EOF from 127.5.78.1:33993 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 20:46:58 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:58 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:58.869343 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:58.859876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:45793)
0926 20:46:58.860435 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:58.860451 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:58.860588 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:58.860977 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:58.860982 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:58.861412 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:58.861575 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:58.861584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.863095 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.863099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:58.864052 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:58.864064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.864873 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.864878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:58.864998 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:58.865583 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:58.865600 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:58.869094 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":315,"threads_started":1}
W20260926 20:46:58.869493 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:58.859964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:45793 (local address 127.5.78.62:32769)
0926 20:46:58.860365 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:58.860371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:58.860532 (+ 161us) server_negotiation.cc:407] Connection header received
0926 20:46:58.860719 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:58.860724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:58.860800 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:58.860906 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:58.861742 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.862934 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.864203 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:58.864687 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:58.864789 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:58.869278 (+ 4489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:45793: BlockingRecv error: recv got EOF from 127.5.78.2:45793 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 20:46:59 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:59 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:59.366665 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:59.357988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:38699)
0926 20:46:59.358578 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:59.358594 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:59.358757 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:59.359145 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:59.359150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:59.359664 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:59.359811 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:59.359821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.360875 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.360881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:59.361944 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:59.361957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.362755 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.362761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:59.362928 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:59.363533 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:59.363549 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:59.366332 (+ 2783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":324,"threads_started":1}
W20260926 20:46:59.366856 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:59.358232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:38699 (local address 127.5.78.62:32769)
0926 20:46:59.358454 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:59.358461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:59.358700 (+ 239us) server_negotiation.cc:407] Connection header received
0926 20:46:59.358905 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:59.358910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:59.358973 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:59.359079 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:59.359975 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.360731 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.362102 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.362596 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.362734 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:59.366585 (+ 3851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:38699: BlockingRecv error: recv got EOF from 127.5.78.3:38699 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 20:46:59 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:59 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:59.413960 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:59.405535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:45603)
0926 20:46:59.406089 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:59.406106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:59.406281 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:59.406636 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:59.406640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:59.407000 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:59.407154 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:59.407162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.408189 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.408196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:59.409089 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:59.409100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.409845 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.409850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:59.410023 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:59.410687 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:59.410706 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:59.413596 (+ 2890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":314,"threads_started":1}
W20260926 20:46:59.414096 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:59.405632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:45603 (local address 127.5.78.62:32769)
0926 20:46:59.405858 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:59.405865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:59.406213 (+ 348us) server_negotiation.cc:407] Connection header received
0926 20:46:59.406436 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:59.406440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:59.406500 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:59.406609 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:59.407301 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.408036 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.409230 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.409678 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.409816 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:59.413833 (+ 4017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:45603: BlockingRecv error: recv got EOF from 127.5.78.1:45603 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 20:46:59 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:46:59 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:46:59.882449 14712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:46:59.870366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:33607)
0926 20:46:59.870932 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0926 20:46:59.870954 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:46:59.871140 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:46:59.871630 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:46:59.871638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:46:59.872178 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:46:59.872408 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:59.872423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.873849 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.873854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:59.874784 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:46:59.874798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.875537 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.875542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:46:59.875668 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:59.876439 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:46:59.876457 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:46:59.882031 (+ 5574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":292,"threads_started":1}
W20260926 20:46:59.882917 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:46:59.870554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:33607 (local address 127.5.78.62:32769)
0926 20:46:59.870762 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:46:59.870768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:46:59.871071 (+ 303us) server_negotiation.cc:407] Connection header received
0926 20:46:59.871292 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:46:59.871297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:46:59.871388 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:46:59.871514 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:46:59.872638 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.873673 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.874964 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:46:59.875393 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:46:59.875512 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:46:59.882334 (+ 6822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:33607: BlockingRecv error: recv got EOF from 127.5.78.2:33607 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 20:47:00 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:00 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:00.376602 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:00.367674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:60285)
0926 20:47:00.368227 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:00.368243 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:00.368414 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:00.368808 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:00.368812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:00.369182 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:00.369364 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:00.369374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.370401 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.370462 (+ 61us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:00.371365 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:00.371376 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.372121 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.372126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:00.372306 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:00.372984 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:00.373002 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:00.376207 (+ 3205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":305,"threads_started":1}
W20260926 20:47:00.376699 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:00.367787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:60285 (local address 127.5.78.62:32769)
0926 20:47:00.368074 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:00.368081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:00.368339 (+ 258us) server_negotiation.cc:407] Connection header received
0926 20:47:00.368598 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:00.368602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:00.368666 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:00.368774 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:00.369526 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.370259 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.371534 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.371949 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.372093 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:00.376430 (+ 4337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:60285: BlockingRecv error: recv got EOF from 127.5.78.3:60285 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 26 20:47:00 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:00 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:00.424065 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:00.415183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:55693)
0926 20:47:00.415741 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:00.415758 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:00.415873 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:00.416228 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:00.416235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:00.416614 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:00.416762 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:00.416770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.417817 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.417821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:00.418828 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:00.418841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.419652 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.419657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:00.419825 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:00.420611 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:00.420640 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:00.423798 (+ 3158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":298,"threads_started":1}
W20260926 20:47:00.424397 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:00.415202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:55693 (local address 127.5.78.62:32769)
0926 20:47:00.415401 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:00.415407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:00.415828 (+ 421us) server_negotiation.cc:407] Connection header received
0926 20:47:00.415965 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:00.415969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:00.416055 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:00.416154 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:00.416917 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.417656 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.418999 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.419430 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.419561 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:00.424081 (+ 4520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:55693: BlockingRecv error: recv got EOF from 127.5.78.1:55693 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 20:47:00 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:00 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:00.897614 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:00.883572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:53489)
0926 20:47:00.884123 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:00.884139 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:00.884261 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:00.884572 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:00.884576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:00.884932 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:00.885127 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:00.885136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.886309 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.886314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:00.887371 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:00.887385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.888140 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.888145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:00.888281 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:00.894199 (+ 5918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:00.894220 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:00.897372 (+ 3152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":311,"threads_started":1}
W20260926 20:47:00.897850 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:00.883718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:53489 (local address 127.5.78.62:32769)
0926 20:47:00.883889 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:00.883893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:00.884254 (+ 361us) server_negotiation.cc:407] Connection header received
0926 20:47:00.884371 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:00.884375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:00.884440 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:00.884532 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:00.885291 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.886114 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.887535 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:00.887982 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:00.888093 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:00.897615 (+ 9522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:53489: BlockingRecv error: recv got EOF from 127.5.78.2:53489 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 20:47:01 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:01 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:01.390849 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:01.377868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:58105)
0926 20:47:01.378448 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:01.378465 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:01.378603 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:01.379002 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:01.379005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:01.379378 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:01.379545 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:01.379553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.380706 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.380712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:01.381890 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:01.381906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.382929 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.382933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:01.383062 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:01.387482 (+ 4420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:01.387503 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:01.390598 (+ 3095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":295,"threads_started":1}
W20260926 20:47:01.391175 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:01.378037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:58105 (local address 127.5.78.62:32769)
0926 20:47:01.378262 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:01.378268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:01.378580 (+ 312us) server_negotiation.cc:407] Connection header received
0926 20:47:01.378724 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:01.378728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:01.378805 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:01.378906 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:01.379726 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.380531 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.382061 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.382780 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.382892 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:01.390892 (+ 8000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:58105: BlockingRecv error: recv got EOF from 127.5.78.3:58105 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 26 20:47:01 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:01 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:01.433745 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:01.425120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:57301)
0926 20:47:01.425754 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:01.425770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:01.425934 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:01.426298 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:01.426306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:01.426749 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:01.426901 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:01.426909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.427941 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.427949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:01.428875 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:01.428887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.429568 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.429573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:01.429749 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:01.430450 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:01.430473 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:01.433392 (+ 2919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":305,"threads_started":1}
W20260926 20:47:01.433835 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:01.425320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:57301 (local address 127.5.78.62:32769)
0926 20:47:01.425600 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:01.425606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:01.425862 (+ 256us) server_negotiation.cc:407] Connection header received
0926 20:47:01.426064 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:01.426069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:01.426138 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:01.426242 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:01.427058 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.427790 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.429018 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.429420 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.429556 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:01.433600 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:57301: BlockingRecv error: recv got EOF from 127.5.78.1:57301 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Sep 26 20:47:01 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:01 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:01.907635 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:01.898809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:41791)
0926 20:47:01.899428 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:01.899451 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:01.899589 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:01.899961 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:01.899966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:01.900516 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:01.900722 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:01.900736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.901813 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.901817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:01.902843 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:01.902854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.903601 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.903605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:01.903723 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:01.904480 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:01.904498 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:01.907400 (+ 2902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":407,"threads_started":1}
W20260926 20:47:01.907917 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:01.899010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:41791 (local address 127.5.78.62:32769)
0926 20:47:01.899209 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:01.899215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:01.899541 (+ 326us) server_negotiation.cc:407] Connection header received
0926 20:47:01.899698 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:01.899704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:01.899802 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:01.899887 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:01.900874 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.901679 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.903001 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:01.903458 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:01.903563 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:01.907607 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:41791: BlockingRecv error: recv got EOF from 127.5.78.2:41791 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 20:47:02 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:02 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:02.400790 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:02.391947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:54227)
0926 20:47:02.392465 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:02.392481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:02.392641 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:02.393086 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:02.393090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:02.393473 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:02.393642 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:02.393650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.394699 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.394703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:02.395931 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:02.395946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.396746 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.396751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:02.396873 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:02.397511 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:02.397529 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:02.400548 (+ 3019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":274,"threads_started":1}
W20260926 20:47:02.401026 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:02.392028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:54227 (local address 127.5.78.62:32769)
0926 20:47:02.392318 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:02.392324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:02.392567 (+ 243us) server_negotiation.cc:407] Connection header received
0926 20:47:02.392838 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:02.392843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:02.392921 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:02.393016 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:02.393796 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.394545 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.396112 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.396581 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.396731 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:02.400797 (+ 4066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:54227: BlockingRecv error: recv got EOF from 127.5.78.3:54227 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 26 20:47:02 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:02 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:02.444173 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:02.434977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:33853)
0926 20:47:02.435576 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:02.435595 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:02.435759 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:02.436126 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:02.436130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:02.436529 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:02.436714 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:02.436722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.437704 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.437708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:02.438707 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:02.438720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.439635 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.439640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:02.439823 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:02.440634 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:02.440661 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:02.443941 (+ 3280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":364,"threads_started":1}
W20260926 20:47:02.444367 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:02.435107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:33853 (local address 127.5.78.62:32769)
0926 20:47:02.435291 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:02.435297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:02.435727 (+ 430us) server_negotiation.cc:407] Connection header received
0926 20:47:02.435879 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:02.435883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:02.435946 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:02.436074 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:02.436855 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.437583 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.438892 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.439461 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.439620 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:02.444124 (+ 4504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:33853: BlockingRecv error: recv got EOF from 127.5.78.1:33853 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 20:47:02 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:02 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:02.917413 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:02.908665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:54679)
0926 20:47:02.909232 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:02.909257 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:02.909384 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:02.909776 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:02.909780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:02.910213 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:02.910475 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:02.910487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.911561 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.911565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:02.912473 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:02.912484 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.913227 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.913232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:02.913368 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:02.914024 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:02.914042 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:02.917079 (+ 3037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":318,"threads_started":1}
W20260926 20:47:02.917618 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:02.908851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:54679 (local address 127.5.78.62:32769)
0926 20:47:02.909035 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:02.909039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:02.909354 (+ 315us) server_negotiation.cc:407] Connection header received
0926 20:47:02.909488 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:02.909492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:02.909562 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:02.909709 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:02.910649 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.911405 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.912630 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:02.913051 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:02.913167 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:02.917352 (+ 4185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:54679: BlockingRecv error: recv got EOF from 127.5.78.2:54679 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 20:47:03 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:03 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:03.410789 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:03.401904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:55167)
0926 20:47:03.402510 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:03.402526 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:03.402699 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:03.403081 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:03.403085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:03.403461 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:03.403611 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:03.403620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.404741 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.404745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:03.405657 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:03.405668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.406386 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.406391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:03.406636 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:03.407248 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:03.407265 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:03.410433 (+ 3168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":320,"threads_started":1}
W20260926 20:47:03.411100 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:03.401949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:55167 (local address 127.5.78.62:32769)
0926 20:47:03.402204 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:03.402213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:03.402718 (+ 505us) server_negotiation.cc:407] Connection header received
0926 20:47:03.402795 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:03.402800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:03.402912 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:03.403021 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:03.403758 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.404601 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.405793 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.406217 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.406317 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:03.410745 (+ 4428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:55167: BlockingRecv error: recv got EOF from 127.5.78.3:55167 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 26 20:47:03 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:03 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:03.454706 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:03.445362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:39741)
0926 20:47:03.446004 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:03.446022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:03.446157 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:03.446573 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:03.446577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:03.447002 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:03.447213 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:03.447224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.448599 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.448604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:03.449644 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:03.449660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.450527 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.450533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:03.450674 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:03.451275 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:03.451295 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:03.454307 (+ 3012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":286,"threads_started":1}
W20260926 20:47:03.454942 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:03.445482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:39741 (local address 127.5.78.62:32769)
0926 20:47:03.445709 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:03.445714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:03.446170 (+ 456us) server_negotiation.cc:407] Connection header received
0926 20:47:03.446265 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:03.446269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:03.446337 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:03.446456 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:03.447411 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.448414 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.449812 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.450247 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.450352 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:03.454677 (+ 4325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:39741: BlockingRecv error: recv got EOF from 127.5.78.1:39741 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 26 20:47:03 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:03 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:03.928028 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:03.918700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:36943)
0926 20:47:03.919253 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:03.919270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:03.919391 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:03.919785 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:03.919789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:03.920317 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:03.920493 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:03.920501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.921866 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.921894 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:03.923184 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:03.923199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.924056 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.924061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:03.924174 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:03.924761 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:03.924779 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:03.927765 (+ 2986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":245,"threads_started":1}
W20260926 20:47:03.928324 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:03.918806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36943 (local address 127.5.78.62:32769)
0926 20:47:03.919084 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:03.919090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:03.919356 (+ 266us) server_negotiation.cc:407] Connection header received
0926 20:47:03.919512 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:03.919518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:03.919583 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:03.919670 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:03.920713 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.921703 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.923384 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:03.923906 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:03.924011 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:03.927997 (+ 3986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:36943: BlockingRecv error: recv got EOF from 127.5.78.2:36943 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 26 20:47:04 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:04 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:04.424054 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:04.412002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:57205)
0926 20:47:04.413481 (+ 1479us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:04.413503 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:04.413670 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:04.414127 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:04.414131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:04.414709 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:04.414876 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:04.414885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.416651 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.416657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:04.418123 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:04.418136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.419340 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.419345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:04.419457 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:04.420104 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:04.420122 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:04.423611 (+ 3489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1318,"thread_start_us":1060,"threads_started":1}
W20260926 20:47:04.424116 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:04.412843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:57205 (local address 127.5.78.62:32769)
0926 20:47:04.413032 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:04.413039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:04.413634 (+ 595us) server_negotiation.cc:407] Connection header received
0926 20:47:04.413840 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:04.413848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:04.413947 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:04.414068 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:04.415056 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.416456 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.418286 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.419183 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.419340 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:04.423924 (+ 4584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:57205: BlockingRecv error: recv got EOF from 127.5.78.3:57205 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 20:47:04 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:04 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:04.464335 14726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:04.455749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:40895)
0926 20:47:04.456320 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:04.456337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:04.456454 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:04.456881 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:04.456885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:04.457302 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:04.457454 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:04.457463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.458599 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.458604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:04.459498 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:04.459508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.460208 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.460213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:04.460326 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:04.460912 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:04.460929 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:04.464069 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":344,"threads_started":1}
W20260926 20:47:04.464524 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:04.455967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:40895 (local address 127.5.78.62:32769)
0926 20:47:04.456156 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:04.456160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:04.456413 (+ 253us) server_negotiation.cc:407] Connection header received
0926 20:47:04.456552 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:04.456555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:04.456633 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:04.456719 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:04.457622 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.458349 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.459659 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.460054 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.460149 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:04.464284 (+ 4135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:40895: BlockingRecv error: recv got EOF from 127.5.78.1:40895 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 20:47:04 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:04 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:04.937783 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:04.929093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:32813)
0926 20:47:04.929673 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:04.929698 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:04.929843 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:04.930166 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:04.930170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:04.930673 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:04.930822 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:04.930831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.931970 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.931976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:04.932900 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:04.932911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.933719 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.933727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:04.933904 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:04.934579 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:04.934597 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:04.937553 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":299,"threads_started":1}
W20260926 20:47:04.938072 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:04.929144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:32813 (local address 127.5.78.62:32769)
0926 20:47:04.929336 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:04.929344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:04.929776 (+ 432us) server_negotiation.cc:407] Connection header received
0926 20:47:04.929982 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:04.929986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:04.930044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:04.930143 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:04.930967 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.931822 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.933046 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:04.933495 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:04.933657 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:04.937776 (+ 4119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:32813: BlockingRecv error: recv got EOF from 127.5.78.2:32813 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 20:47:05 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:05 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:05.433719 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:05.425383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:48475)
0926 20:47:05.425911 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:05.425928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:05.426045 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:05.426473 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:05.426477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:05.426887 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:05.427039 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:05.427049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.428103 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.428108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:05.429046 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:05.429059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.429796 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.429801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:05.429944 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:05.430626 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:05.430644 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:05.433488 (+ 2844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":283,"threads_started":1}
W20260926 20:47:05.434083 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:05.425486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:48475 (local address 127.5.78.62:32769)
0926 20:47:05.425683 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:05.425687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:05.426023 (+ 336us) server_negotiation.cc:407] Connection header received
0926 20:47:05.426218 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:05.426223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:05.426286 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:05.426385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:05.427217 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.427952 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.429213 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.429627 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.429744 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:05.433741 (+ 3997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:48475: BlockingRecv error: recv got EOF from 127.5.78.3:48475 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 20:47:05 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:05 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:05.474105 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:05.465364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:37609)
0926 20:47:05.465950 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:05.465970 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:05.466093 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:05.466536 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:05.466540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:05.466949 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:05.467143 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:05.467153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.468210 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.468214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:05.469195 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:05.469209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.470094 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.470098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:05.470228 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:05.470969 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:05.470988 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:05.473858 (+ 2870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":297,"threads_started":1}
W20260926 20:47:05.474552 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:05.465433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:37609 (local address 127.5.78.62:32769)
0926 20:47:05.465645 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:05.465650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:05.466071 (+ 421us) server_negotiation.cc:407] Connection header received
0926 20:47:05.466198 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:05.466202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:05.466287 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:05.466382 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:05.467315 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.468075 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.469397 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.469922 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.470083 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:05.474228 (+ 4145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:37609: BlockingRecv error: recv got EOF from 127.5.78.1:37609 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 20:47:05 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:05 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:05.947502 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:05.938796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:54441)
0926 20:47:05.939291 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:05.939306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:05.939480 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:05.939839 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:05.939845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:05.940238 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:05.940479 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:05.940493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.941653 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.941660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:05.942872 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:05.942888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.943798 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.943803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:05.943942 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:05.944499 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:05.944517 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:05.947265 (+ 2748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":257,"threads_started":1}
W20260926 20:47:05.947690 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:05.938852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:54441 (local address 127.5.78.62:32769)
0926 20:47:05.939107 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:05.939114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:05.939410 (+ 296us) server_negotiation.cc:407] Connection header received
0926 20:47:05.939627 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:05.939631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:05.939690 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:05.939791 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:05.940631 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.941471 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.943024 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:05.943626 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:05.943731 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:05.947454 (+ 3723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:54441: BlockingRecv error: recv got EOF from 127.5.78.2:54441 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Sep 26 20:47:06 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:06 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:06.456106 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:06.434844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:51013)
0926 20:47:06.435376 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:06.435393 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:06.435572 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:06.435952 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:06.435956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:06.436316 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:06.436493 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:06.436502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.437497 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.437500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:06.438480 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:06.438491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.439171 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.439175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:06.439297 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:06.452599 (+ 13302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:06.452622 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:06.455755 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":285,"threads_started":1}
W20260926 20:47:06.456399 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:06.434917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:51013 (local address 127.5.78.62:32769)
0926 20:47:06.435143 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:06.435152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:06.435489 (+ 337us) server_negotiation.cc:407] Connection header received
0926 20:47:06.435732 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:06.435736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:06.435820 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:06.435906 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:06.436648 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.437382 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.438621 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.439031 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.439130 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:06.456120 (+ 16990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:51013: BlockingRecv error: recv got EOF from 127.5.78.3:51013 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 20:47:06 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:06 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:06.484608 14732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:06.475384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:37021)
0926 20:47:06.475999 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:06.476016 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:06.476182 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:06.476534 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:06.476539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:06.476935 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:06.477069 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:06.477077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.478099 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.478106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:06.479390 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:06.479402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.480144 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.480149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:06.480328 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:06.480956 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:06.480989 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:06.484384 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":332,"threads_started":1}
W20260926 20:47:06.484975 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:06.475517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:37021 (local address 127.5.78.62:32769)
0926 20:47:06.475778 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:06.475786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:06.476115 (+ 329us) server_negotiation.cc:407] Connection header received
0926 20:47:06.476319 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:06.476323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:06.476398 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:06.476506 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:06.477227 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.477958 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.479535 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.479988 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.480128 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:06.484601 (+ 4473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:37021: BlockingRecv error: recv got EOF from 127.5.78.1:37021 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 26 20:47:06 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:06 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:06.957975 14733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:06.948538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:57203)
0926 20:47:06.949146 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:06.949163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:06.949345 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:06.949740 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:06.949745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:06.950175 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:06.950326 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:06.950336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.951673 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.951677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:06.952837 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:06.952860 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.953621 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.953625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:06.953747 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:06.954443 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:06.954462 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:06.957741 (+ 3279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":348,"threads_started":1}
W20260926 20:47:06.958299 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:06.948687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:57203 (local address 127.5.78.62:32769)
0926 20:47:06.948959 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:06.948980 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:06.949269 (+ 289us) server_negotiation.cc:407] Connection header received
0926 20:47:06.949497 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:06.949504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:06.949580 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:06.949706 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:06.950543 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.951525 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.952999 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:06.953473 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:06.953591 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:06.957984 (+ 4393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:57203: BlockingRecv error: recv got EOF from 127.5.78.2:57203 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Sep 26 20:47:07 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:07 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:07.466015 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:07.457252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:52045)
0926 20:47:07.457811 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:07.457833 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:07.458040 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:07.458439 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:07.458444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:07.458864 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:07.459015 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:07.459025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.460045 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.460051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:07.460991 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:07.461003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.461951 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.461958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:07.462128 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:07.462946 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:07.462971 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:07.465788 (+ 2817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":277,"threads_started":1}
W20260926 20:47:07.466290 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:07.457566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:52045 (local address 127.5.78.62:32769)
0926 20:47:07.457809 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:07.457816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:07.457939 (+ 123us) server_negotiation.cc:407] Connection header received
0926 20:47:07.458177 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:07.458183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:07.458266 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:07.458375 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:07.459178 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.459896 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.461143 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.461734 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.461883 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:07.466021 (+ 4138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:52045: BlockingRecv error: recv got EOF from 127.5.78.3:52045 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Sep 26 20:47:07 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:07 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:07.494380 14735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:07.485710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:50011)
0926 20:47:07.486272 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:07.486288 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:07.486481 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:07.486882 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:07.486890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:07.487458 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:07.487687 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:07.487700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.488915 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.488919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:07.489822 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:07.489833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.490545 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.490549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:07.490670 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:07.491276 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:07.491294 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:07.494139 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":331,"threads_started":1}
W20260926 20:47:07.494831 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:07.485810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:50011 (local address 127.5.78.62:32769)
0926 20:47:07.486057 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:07.486065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:07.486392 (+ 327us) server_negotiation.cc:407] Connection header received
0926 20:47:07.486634 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:07.486639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:07.486725 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:07.486814 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:07.487870 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.488785 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.489946 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.490389 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.490514 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:07.494346 (+ 3832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:50011: BlockingRecv error: recv got EOF from 127.5.78.1:50011 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 20:47:07 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:07 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:07.967473 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:07.958910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:37185)
0926 20:47:07.959370 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:07.959385 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:07.959522 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:07.959812 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:07.959815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:07.960180 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:07.960349 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:07.960357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.961494 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.961498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:07.962498 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:07.962510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.963289 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.963293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:07.963411 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:07.964074 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:07.964094 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:07.967111 (+ 3017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":258,"threads_started":1}
W20260926 20:47:07.967550 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:07.959017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:37185 (local address 127.5.78.62:32769)
0926 20:47:07.959280 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:07.959287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:07.959479 (+ 192us) server_negotiation.cc:407] Connection header received
0926 20:47:07.959623 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:07.959627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:07.959690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:07.959773 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:07.960501 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.961329 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.962640 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:07.963119 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:07.963236 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:07.967319 (+ 4083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:37185: BlockingRecv error: recv got EOF from 127.5.78.2:37185 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 26 20:47:08 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:08 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:08.475924 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:08.467000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:58983)
0926 20:47:08.467554 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:08.467572 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:08.467758 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:08.468147 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:08.468152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:08.468583 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:08.468776 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:08.468786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.469831 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.469838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:08.471035 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:08.471048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.471902 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.471908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:08.472054 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:08.472715 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:08.472734 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:08.475575 (+ 2841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":285,"threads_started":1}
W20260926 20:47:08.476014 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:08.467181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:58983 (local address 127.5.78.62:32769)
0926 20:47:08.467419 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:08.467425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:08.467666 (+ 241us) server_negotiation.cc:407] Connection header received
0926 20:47:08.467893 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:08.467899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:08.467977 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:08.468085 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:08.468946 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.469692 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.471192 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.471679 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.471798 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:08.475789 (+ 3991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:58983: BlockingRecv error: recv got EOF from 127.5.78.3:58983 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Sep 26 20:47:08 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:08 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:08.504328 14738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:08.495538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:55455)
0926 20:47:08.496151 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:08.496175 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:08.496332 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:08.496703 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:08.496706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:08.497252 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:08.497429 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:08.497437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.498537 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.498542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:08.499477 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:08.499489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.500283 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.500287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:08.500416 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:08.501020 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:08.501039 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:08.503981 (+ 2942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":287,"threads_started":1}
W20260926 20:47:08.504602 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:08.495671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:55455 (local address 127.5.78.62:32769)
0926 20:47:08.495833 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:08.495837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:08.496257 (+ 420us) server_negotiation.cc:407] Connection header received
0926 20:47:08.496450 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:08.496454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:08.496529 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:08.496644 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:08.497579 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.498289 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.499649 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.500140 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.500246 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:08.504203 (+ 3957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:55455: BlockingRecv error: recv got EOF from 127.5.78.1:55455 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 20:47:08 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:08 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:08.977743 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:08.968583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:43271)
0926 20:47:08.969163 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:08.969181 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:08.969378 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:08.969752 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:08.969757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:08.970150 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:08.970303 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:08.970311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.971349 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.971356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:08.972264 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:08.972276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.973035 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.973042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:08.973245 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:08.974039 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:08.974065 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:08.977401 (+ 3336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":353,"threads_started":1}
W20260926 20:47:08.977905 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:08.968695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:43271 (local address 127.5.78.62:32769)
0926 20:47:08.969005 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:08.969013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:08.969293 (+ 280us) server_negotiation.cc:407] Connection header received
0926 20:47:08.969518 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:08.969529 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:08.969593 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:08.969704 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:08.970505 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.971194 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.972414 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:08.972880 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:08.973021 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:08.977650 (+ 4629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:43271: BlockingRecv error: recv got EOF from 127.5.78.2:43271 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Sep 26 20:47:09 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:09 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:09.486016 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:09.476945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:48047)
0926 20:47:09.477513 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:09.477528 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:09.477695 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:09.478034 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:09.478039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:09.478482 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:09.478638 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:09.478647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.479664 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.479671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:09.480590 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:09.480601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.481362 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.481369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:09.481561 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:09.482203 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:09.482221 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:09.485781 (+ 3560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":309,"threads_started":1}
W20260926 20:47:09.486253 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:09.477011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:48047 (local address 127.5.78.62:32769)
0926 20:47:09.477306 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:09.477315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:09.477626 (+ 311us) server_negotiation.cc:407] Connection header received
0926 20:47:09.477836 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:09.477842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:09.477902 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:09.478009 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:09.478788 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.479515 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.480735 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.481175 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.481333 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:09.486001 (+ 4668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:48047: BlockingRecv error: recv got EOF from 127.5.78.3:48047 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Sep 26 20:47:09 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:09 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:09.514842 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:09.505497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:50285)
0926 20:47:09.505985 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:09.506002 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:09.506122 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:09.506537 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:09.506540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:09.506984 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:09.507183 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:09.507193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.508318 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.508324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:09.509884 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:09.509899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.510742 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.510750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:09.510931 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:09.511553 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:09.511573 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:09.514594 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":258,"threads_started":1}
W20260926 20:47:09.515019 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:09.505559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:50285 (local address 127.5.78.62:32769)
0926 20:47:09.505797 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:09.505803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:09.506087 (+ 284us) server_negotiation.cc:407] Connection header received
0926 20:47:09.506221 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:09.506225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:09.506298 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:09.506387 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:09.507322 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.508164 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.510042 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.510537 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.510698 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:09.514805 (+ 4107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:50285: BlockingRecv error: recv got EOF from 127.5.78.1:50285 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 20:47:09 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:09 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:09.988722 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:09.979084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:53387)
0926 20:47:09.979727 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:09.979749 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:09.979923 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:09.980381 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:09.980387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:09.980906 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:09.981132 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:09.981148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.982359 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.982367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:09.983353 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:09.983365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.984134 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.984140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:09.984338 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:09.985063 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:09.985081 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:09.988469 (+ 3388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":332,"threads_started":1}
W20260926 20:47:09.989117 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:09.979442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:53387 (local address 127.5.78.62:32769)
0926 20:47:09.979694 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:09.979698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:09.979940 (+ 242us) server_negotiation.cc:407] Connection header received
0926 20:47:09.980036 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:09.980043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:09.980159 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:09.980353 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:09.981322 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.982199 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.983519 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:09.983973 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:09.984122 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:09.988762 (+ 4640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:53387: BlockingRecv error: recv got EOF from 127.5.78.2:53387 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 26 20:47:10 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:10 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:10.496091 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:10.487052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:52761)
0926 20:47:10.487715 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:10.487732 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:10.487901 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:10.488254 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:10.488258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:10.488698 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:10.488860 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:10.488869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.489952 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.489959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:10.490922 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:10.490935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.491710 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.491717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:10.491892 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:10.492673 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:10.492701 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:10.495830 (+ 3129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":351,"threads_started":1}
W20260926 20:47:10.496246 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:10.487187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:52761 (local address 127.5.78.62:32769)
0926 20:47:10.487429 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:10.487438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:10.487845 (+ 407us) server_negotiation.cc:407] Connection header received
0926 20:47:10.488054 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:10.488058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:10.488120 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:10.488225 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:10.489052 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.489797 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.491080 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.491512 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.491659 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:10.496031 (+ 4372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:52761: BlockingRecv error: recv got EOF from 127.5.78.3:52761 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 26 20:47:10 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:10 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:10.524556 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:10.515806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:43511)
0926 20:47:10.516307 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:10.516324 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:10.516463 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:10.516908 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:10.516911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:10.517264 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:10.517523 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:10.517533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.518873 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.518877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:10.519909 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:10.519921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.520628 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.520632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:10.520819 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:10.521413 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:10.521430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:10.524290 (+ 2860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":272,"threads_started":1}
W20260926 20:47:10.524817 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:10.515877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:43511 (local address 127.5.78.62:32769)
0926 20:47:10.516114 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:10.516118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:10.516445 (+ 327us) server_negotiation.cc:407] Connection header received
0926 20:47:10.516585 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:10.516590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:10.516723 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:10.516824 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:10.517664 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.518738 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.520056 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.520494 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.520590 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:10.524547 (+ 3957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:43511: BlockingRecv error: recv got EOF from 127.5.78.1:43511 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Sep 26 20:47:10 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:10 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:10.999321 14745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:10.989737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:39283)
0926 20:47:10.990270 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:10.990287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:10.990478 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:10.990840 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:10.990845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:10.991221 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:10.991392 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:10.991400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.992440 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.992445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:10.993379 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:10.993390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.994156 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.994163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:10.994347 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:10.995168 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:10.995186 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:10.998908 (+ 3722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":265,"threads_started":1}
W20260926 20:47:10.999439 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:10.989820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:39283 (local address 127.5.78.62:32769)
0926 20:47:10.990016 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:10.990023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:10.990385 (+ 362us) server_negotiation.cc:407] Connection header received
0926 20:47:10.990630 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:10.990638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:10.990724 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:10.990814 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:10.991544 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.992296 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.993546 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:10.993994 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:10.994130 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:10.999174 (+ 5044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:39283: BlockingRecv error: recv got EOF from 127.5.78.2:39283 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 20:47:11 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:11 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.3@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:11.511310 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:11.497251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.3:54549)
0926 20:47:11.497836 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:11.497855 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:11.497999 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:11.498350 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:11.498357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:11.498743 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:11.498996 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:11.499007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.500001 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.500016 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:11.501054 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:11.501068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.501815 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.501820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:11.501955 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:11.507219 (+ 5264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:11.507242 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:11.510943 (+ 3701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":288,"threads_started":1}
W20260926 20:47:11.511482 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:11.497336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:54549 (local address 127.5.78.62:32769)
0926 20:47:11.497546 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:11.497555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:11.497951 (+ 396us) server_negotiation.cc:407] Connection header received
0926 20:47:11.498128 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:11.498132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:11.498204 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:11.498277 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:11.499103 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.499829 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.501252 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.501659 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.501762 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:11.511198 (+ 9436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.3:54549: BlockingRecv error: recv got EOF from 127.5.78.3:54549 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 20:47:11 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:11 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.1@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:11.534046 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:11.525696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.1:39267)
0926 20:47:11.526254 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:11.526271 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:11.526388 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:11.526794 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:11.526798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:11.527202 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:11.527392 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:11.527401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.528388 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.528392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:11.529348 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:11.529361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.530056 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.530061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:11.530178 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:11.530911 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:11.530928 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:11.533791 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":315,"threads_started":1}
W20260926 20:47:11.534240 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:11.525737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:39267 (local address 127.5.78.62:32769)
0926 20:47:11.525907 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:11.525912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:11.526342 (+ 430us) server_negotiation.cc:407] Connection header received
0926 20:47:11.526547 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:11.526551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:11.526634 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:11.526720 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:11.527527 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.528238 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.529477 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:11.529931 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:11.530028 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:11.534008 (+ 3980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.1:39267: BlockingRecv error: recv got EOF from 127.5.78.1:39267 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 20:47:12 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:47:12 dist-test-slave-z7j8 krb5kdc[13703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.78.2@KRBTEST.COM for oryx/127.5.78.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:47:12.009404 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:12.000540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32769 (local address 127.5.78.2:36293)
0926 20:47:12.001152 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:12.001170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:12.001274 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:12.001597 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:12.001601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:12.002004 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:12.002190 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:12.002199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.003319 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.003323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:12.004306 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:12.004319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.005109 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.005114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:12.005227 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:12.005863 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:12.005881 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:12.009024 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.78.62:32769: Server oryx/127.5.78.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":342,"threads_started":1}
W20260926 20:47:12.009552 14439 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:12.000735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:36293 (local address 127.5.78.62:32769)
0926 20:47:12.000904 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:12.000908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:12.001227 (+ 319us) server_negotiation.cc:407] Connection header received
0926 20:47:12.001374 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:12.001378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:12.001438 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:12.001511 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:12.002316 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.003076 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.004466 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.004943 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.005049 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:12.009301 (+ 4252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.78.2:36293: BlockingRecv error: recv got EOF from 127.5.78.2:36293 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:47:12.325129 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14290
I20260926 20:47:12.338876 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:12.326306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.5.78.1:44489)
0926 20:47:12.326918 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:12.326932 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:12.326973 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:47:12.327064 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:12.327072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:12.327310 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:12.327577 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:12.329248 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.330371 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.331869 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.332534 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.332710 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:12.335423 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:12.335435 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:12.335459 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:12.335494 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:12.338178 (+ 2684us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.78.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 20:47:12.338555 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41020: Request ticket server kudu/127.5.78.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":276,"threads_started":1}
W20260926 20:47:12.339001 14750 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41020: Request ticket server kudu/127.5.78.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 20:47:12.339138 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14290: Not authorized: Client connection negotiation failed: client connection to 127.5.78.1:44489: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.78.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 20:47:12.339371 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14290
I20260926 20:47:12.356086 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14441
I20260926 20:47:12.369308 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:12.357385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.5.78.2:40565)
0926 20:47:12.358059 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:12.358066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:12.358097 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:47:12.358176 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:12.358182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:12.358481 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:12.358656 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:12.360120 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.361224 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.362657 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.363265 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.363405 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:12.365971 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:12.365985 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:12.366004 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:12.366052 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:12.368759 (+ 2707us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.78.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 20:47:12.369036 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59406: Request ticket server kudu/127.5.78.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":341,"threads_started":1}
W20260926 20:47:12.369561 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14441: Not authorized: Client connection negotiation failed: client connection to 127.5.78.2:40565: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.78.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 20:47:12.369724 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14441
I20260926 20:47:12.386216 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14572
I20260926 20:47:12.399821 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:12.387324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.5.78.3:38467)
0926 20:47:12.387974 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:12.387981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:12.388008 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:47:12.388074 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:12.388081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:12.388298 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:12.388478 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:12.389853 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.391121 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.392690 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:12.393293 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:12.393412 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:12.396312 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:12.396321 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:12.396336 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:12.396378 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:12.399284 (+ 2906us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.78.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 20:47:12.399565 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37010: Request ticket server kudu/127.5.78.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":383,"threads_started":1}
W20260926 20:47:12.399946 14752 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37010: Request ticket server kudu/127.5.78.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 20:47:12.400032 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14572: Not authorized: Client connection negotiation failed: client connection to 127.5.78.3:38467: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.78.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 20:47:12.400233 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14572
I20260926 20:47:12.416846 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14221
I20260926 20:47:12.418169 14283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:12.531513 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14221
2026-09-26T20:47:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21173 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T20:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:47:12Z Disabled control of system clock
I20260926 20:47:12.610477 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:39191
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:39191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 20:47:12.946159 14757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:12.946494 14757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:12.946578 14757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:12.953891 14757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:47:12.954018 14757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:12.954057 14757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:12.954107 14757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:47:12.954154 14757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:47:12.963675 14757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:39191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:39191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:12.964985 14757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:12.966836 14757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:12.979357 14762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:12.979369 14763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:12.980103 14765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:12.980839 14757 server_base.cc:1034] running on GCE node
I20260926 20:47:12.981724 14757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:12.983351 14757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:12.984609 14757 hybrid_clock.cc:648] HybridClock initialized: now 1790455632984551 us; error 51 us; skew 500 ppm
I20260926 20:47:12.985000 14757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:12.987411 14757 webserver.cc:460] Webserver started at http://127.5.78.62:39269/ using document root <none> and password file <none>
I20260926 20:47:12.988147 14757 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:12.988279 14757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:12.988602 14757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:12.991600 14757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "f0c14756eb8e4d0cb20a8d3b75add76a"
format_stamp: "Formatted at 2026-09-26 20:47:12 on dist-test-slave-z7j8"
server_key: "c01238b1584aa314fdc4847ab0603183"
server_key_iv: "19b55a0bbb06751a08543d2ca8d65b73"
server_key_version: "encryptionkey@0"
I20260926 20:47:12.992408 14757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "f0c14756eb8e4d0cb20a8d3b75add76a"
format_stamp: "Formatted at 2026-09-26 20:47:12 on dist-test-slave-z7j8"
server_key: "c01238b1584aa314fdc4847ab0603183"
server_key_iv: "19b55a0bbb06751a08543d2ca8d65b73"
server_key_version: "encryptionkey@0"
I20260926 20:47:12.997924 14757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:47:13.001608 14771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:13.003374 14757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:47:13.003576 14757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "f0c14756eb8e4d0cb20a8d3b75add76a"
format_stamp: "Formatted at 2026-09-26 20:47:12 on dist-test-slave-z7j8"
server_key: "c01238b1584aa314fdc4847ab0603183"
server_key_iv: "19b55a0bbb06751a08543d2ca8d65b73"
server_key_version: "encryptionkey@0"
I20260926 20:47:13.003834 14757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:13.023699 14757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:13.024612 14757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:13.024923 14757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:13.044144 14757 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:39191
I20260926 20:47:13.044185 14822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:39191 every 8 connection(s)
I20260926 20:47:13.045591 14757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:47:13.050001 14823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:47:13.054044 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14757
I20260926 20:47:13.054262 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:47:13.055088 5432 external_mini_cluster.cc:1428] Setting key ea38129b7260893ed7eeae509a4a1ba9
I20260926 20:47:13.062309 14823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a: Bootstrap starting.
I20260926 20:47:13.067227 14823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a: Neither blocks nor log segments found. Creating new log.
I20260926 20:47:13.068720 14823 log.cc:824] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:47:13.072310 14823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a: No bootstrap required, opened a new log
I20260926 20:47:13.072477 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:13.057385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.5.78.62:39191)
0926 20:47:13.058541 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:13.058555 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:13.058611 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:47:13.060817 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:13.060845 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:13.061400 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:13.061764 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:13.062569 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:13.064887 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:13.066883 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:13.068217 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:13.068425 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:13.069741 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:13.069754 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:13.069781 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:13.070685 (+ 904us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:13.070792 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:13.070813 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:13.070947 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:13.071140 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 20:47:13.071516 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":427,"threads_started":1}
I20260926 20:47:13.080516 14823 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39191 } }
I20260926 20:47:13.080914 14823 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:47:13.081019 14823 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c14756eb8e4d0cb20a8d3b75add76a, State: Initialized, Role: FOLLOWER
I20260926 20:47:13.081560 14823 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39191 } }
I20260926 20:47:13.081800 14823 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:47:13.081939 14823 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:47:13.082121 14823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:47:13.084496 14823 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39191 } }
I20260926 20:47:13.085016 14823 leader_election.cc:302] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0c14756eb8e4d0cb20a8d3b75add76a; no voters:
I20260926 20:47:13.085657 14823 leader_election.cc:288] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:47:13.085861 14828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:47:13.087030 14828 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [term 1 LEADER]: Becoming Leader. State: Replica: f0c14756eb8e4d0cb20a8d3b75add76a, State: Running, Role: LEADER
I20260926 20:47:13.087662 14828 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39191 } }
I20260926 20:47:13.087937 14823 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:47:13.090719 14830 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0c14756eb8e4d0cb20a8d3b75add76a. Latest consensus state: current_term: 1 leader_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39191 } } }
I20260926 20:47:13.091246 14830 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [sys.catalog]: This master's current role is: LEADER
I20260926 20:47:13.090835 14829 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c14756eb8e4d0cb20a8d3b75add76a" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39191 } } }
I20260926 20:47:13.091744 14829 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a [sys.catalog]: This master's current role is: LEADER
I20260926 20:47:13.095355 14837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:47:13.099771 14837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:47:13.108451 14837 catalog_manager.cc:1347] Generated new cluster ID: 8456c91e7c7e436ba30967b1a245626e
I20260926 20:47:13.108582 14837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:47:13.118615 14837 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:47:13.120688 14837 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:47:13.132495 14837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0c14756eb8e4d0cb20a8d3b75add76a: Generated new TSK 0
I20260926 20:47:13.133620 14837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:47:13.146176 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39191
--builtin_ntp_servers=127.5.78.20:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 20:47:13.497921 14847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:13.498313 14847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:13.498539 14847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:13.506318 14847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:13.506528 14847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:13.506731 14847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:47:13.516790 14847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:13.518203 14847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:13.520323 14847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:13.535624 14853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:13.535583 14855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:13.535665 14852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:13.536836 14847 server_base.cc:1034] running on GCE node
I20260926 20:47:13.537583 14847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:13.538756 14847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:13.539994 14847 hybrid_clock.cc:648] HybridClock initialized: now 1790455633539942 us; error 39 us; skew 500 ppm
I20260926 20:47:13.540397 14847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:13.543023 14847 webserver.cc:460] Webserver started at http://127.5.78.1:41135/ using document root <none> and password file <none>
I20260926 20:47:13.543819 14847 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:13.543963 14847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:13.544296 14847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:13.547435 14847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "c6ecc9df00a04ce2aae973f21f3d2800"
format_stamp: "Formatted at 2026-09-26 20:47:13 on dist-test-slave-z7j8"
server_key: "a8709150887c10aaced6f77260ffd90c"
server_key_iv: "03f6ddcfc8c7eb1efb1c0fcb16b65c24"
server_key_version: "encryptionkey@0"
I20260926 20:47:13.548339 14847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "c6ecc9df00a04ce2aae973f21f3d2800"
format_stamp: "Formatted at 2026-09-26 20:47:13 on dist-test-slave-z7j8"
server_key: "a8709150887c10aaced6f77260ffd90c"
server_key_iv: "03f6ddcfc8c7eb1efb1c0fcb16b65c24"
server_key_version: "encryptionkey@0"
I20260926 20:47:13.554039 14847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:47:13.557979 14861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:13.559492 14847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:47:13.559691 14847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "c6ecc9df00a04ce2aae973f21f3d2800"
format_stamp: "Formatted at 2026-09-26 20:47:13 on dist-test-slave-z7j8"
server_key: "a8709150887c10aaced6f77260ffd90c"
server_key_iv: "03f6ddcfc8c7eb1efb1c0fcb16b65c24"
server_key_version: "encryptionkey@0"
I20260926 20:47:13.559942 14847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:13.589478 14847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:13.590612 14847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:13.590914 14847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:13.592094 14847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:13.593741 14847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:13.593838 14847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:13.593995 14847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:13.594090 14847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:13.628921 14847 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:41951
I20260926 20:47:13.629030 14973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:41951 every 8 connection(s)
I20260926 20:47:13.630232 14847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:47:13.632375 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14847
I20260926 20:47:13.632544 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:47:13.633388 5432 external_mini_cluster.cc:1428] Setting key 825abb7aa2563a80e4fcdd584ad5f326
I20260926 20:47:13.636098 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39191
--builtin_ntp_servers=127.5.78.20:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 20:47:13.643277 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:13.633040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:60289 (local address 127.5.78.62:39191)
0926 20:47:13.633514 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:13.633521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:13.634956 (+ 1435us) server_negotiation.cc:407] Connection header received
0926 20:47:13.636903 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:13.636908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:13.636988 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:13.637114 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:13.638164 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:13.638947 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:13.640511 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:13.640929 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:13.641079 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:13.642396 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:13.642402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:13.642427 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:13.642448 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:13.642498 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:13.642510 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:13.642629 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:13.642847 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 20:47:13.642939 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":227,"threads_started":1}
I20260926 20:47:13.644248 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:13.633722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39191 (local address 127.5.78.1:60289)
0926 20:47:13.634804 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:13.634856 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:13.636635 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:13.637354 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:13.637362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:13.637455 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:13.638003 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:13.638019 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:13.639104 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:13.639109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:13.640362 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:13.640373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:13.641143 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:13.641150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:13.641377 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:13.642083 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:47:13.642086 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:13.642125 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:47:13.642196 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:13.642627 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:13.642640 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:13.642650 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 20:47:13.642832 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 20:47:13.643211 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":319,"threads_started":1}
I20260926 20:47:13.645793 14974 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39191
I20260926 20:47:13.646111 14974 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:13.647089 14974 heartbeater.cc:511] Master 127.5.78.62:39191 requested a full tablet report, sending...
I20260926 20:47:13.649518 14788 ts_manager.cc:194] Registered new tserver with Master: c6ecc9df00a04ce2aae973f21f3d2800 (127.5.78.1:41951)
I20260926 20:47:13.651830 14788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.1:60289
W20260926 20:47:13.972215 14978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:13.972520 14978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:13.972602 14978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:13.980016 14978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:13.980129 14978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:13.980233 14978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:47:13.989553 14978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:13.990850 14978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:13.992763 14978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:14.006358 14984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:14.006481 14986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:14.006505 14983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:14.007243 14978 server_base.cc:1034] running on GCE node
I20260926 20:47:14.007941 14978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:14.009007 14978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:14.010273 14978 hybrid_clock.cc:648] HybridClock initialized: now 1790455634010223 us; error 45 us; skew 500 ppm
I20260926 20:47:14.010676 14978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:14.013176 14978 webserver.cc:460] Webserver started at http://127.5.78.2:36029/ using document root <none> and password file <none>
I20260926 20:47:14.013980 14978 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:14.014078 14978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:14.014523 14978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:14.017573 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "6cdd6caaf4e14af49d3e71f48f7a5aee"
format_stamp: "Formatted at 2026-09-26 20:47:14 on dist-test-slave-z7j8"
server_key: "b6ccc073cb1e7af420a654adcdaa503d"
server_key_iv: "bb0e830b7a7e01bc6929a9385336f49d"
server_key_version: "encryptionkey@0"
I20260926 20:47:14.018553 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "6cdd6caaf4e14af49d3e71f48f7a5aee"
format_stamp: "Formatted at 2026-09-26 20:47:14 on dist-test-slave-z7j8"
server_key: "b6ccc073cb1e7af420a654adcdaa503d"
server_key_iv: "bb0e830b7a7e01bc6929a9385336f49d"
server_key_version: "encryptionkey@0"
I20260926 20:47:14.024202 14978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:47:14.028564 14992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:14.030336 14978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:47:14.030601 14978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "6cdd6caaf4e14af49d3e71f48f7a5aee"
format_stamp: "Formatted at 2026-09-26 20:47:14 on dist-test-slave-z7j8"
server_key: "b6ccc073cb1e7af420a654adcdaa503d"
server_key_iv: "bb0e830b7a7e01bc6929a9385336f49d"
server_key_version: "encryptionkey@0"
I20260926 20:47:14.030856 14978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:14.044967 14978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:14.045807 14978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:14.046088 14978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:14.047195 14978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:14.048647 14978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:14.048733 14978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:14.048885 14978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:14.048978 14978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:14.085753 14978 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:42305
I20260926 20:47:14.085788 15104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:42305 every 8 connection(s)
I20260926 20:47:14.087240 14978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:47:14.090821 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 14978
I20260926 20:47:14.090996 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:47:14.091500 5432 external_mini_cluster.cc:1428] Setting key 9ce6ea59e13450de0a8c7e87e7807a17
I20260926 20:47:14.094506 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39191
--builtin_ntp_servers=127.5.78.20:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 20:47:14.101433 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.090220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:37405 (local address 127.5.78.62:39191)
0926 20:47:14.090456 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:14.090464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:14.092346 (+ 1882us) server_negotiation.cc:407] Connection header received
0926 20:47:14.094151 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:14.094157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:14.094244 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:14.094377 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:14.095640 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.096524 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.098356 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.098860 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.099027 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.100591 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:14.100598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:14.100606 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:14.100632 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:14.100676 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:14.100693 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:14.100801 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:14.101092 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 20:47:14.101202 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:47:14.102684 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.090866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39191 (local address 127.5.78.2:37405)
0926 20:47:14.092096 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:14.092214 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:14.093858 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:14.094691 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:14.094705 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:14.094844 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:14.095453 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:14.095473 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.096700 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.096705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:14.098065 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:14.098079 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.099156 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.099166 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:14.099409 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.100213 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:47:14.100216 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:14.100262 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:47:14.100359 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:14.100849 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:14.100861 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:14.100868 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:47:14.101047 (+ 179us) client_negotiation.cc:240] Negotiation successful
0926 20:47:14.101395 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":380,"threads_started":1}
I20260926 20:47:14.104507 15105 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39191
I20260926 20:47:14.104875 15105 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:14.105876 15105 heartbeater.cc:511] Master 127.5.78.62:39191 requested a full tablet report, sending...
I20260926 20:47:14.107662 14788 ts_manager.cc:194] Registered new tserver with Master: 6cdd6caaf4e14af49d3e71f48f7a5aee (127.5.78.2:42305)
I20260926 20:47:14.108551 14788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.2:37405
W20260926 20:47:14.429256 15108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:14.429575 15108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:14.429740 15108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:14.437253 15108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:14.437376 15108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:14.437544 15108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:47:14.447145 15108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:14.448596 15108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:14.450547 15108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:14.464003 15116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:14.463865 15114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:14.463941 15113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:14.464993 15108 server_base.cc:1034] running on GCE node
I20260926 20:47:14.465606 15108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:14.466671 15108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:14.467950 15108 hybrid_clock.cc:648] HybridClock initialized: now 1790455634467900 us; error 44 us; skew 500 ppm
I20260926 20:47:14.468293 15108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:14.470669 15108 webserver.cc:460] Webserver started at http://127.5.78.3:35753/ using document root <none> and password file <none>
I20260926 20:47:14.471359 15108 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:14.471491 15108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:14.471802 15108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:14.474658 15108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "5b64d2ab12784f30baa99c49544ef7f2"
format_stamp: "Formatted at 2026-09-26 20:47:14 on dist-test-slave-z7j8"
server_key: "6291257fb2a0eaedf24d5c279baa678d"
server_key_iv: "e3acdb528af1ff17029daa7a11498ac1"
server_key_version: "encryptionkey@0"
I20260926 20:47:14.475461 15108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "5b64d2ab12784f30baa99c49544ef7f2"
format_stamp: "Formatted at 2026-09-26 20:47:14 on dist-test-slave-z7j8"
server_key: "6291257fb2a0eaedf24d5c279baa678d"
server_key_iv: "e3acdb528af1ff17029daa7a11498ac1"
server_key_version: "encryptionkey@0"
I20260926 20:47:14.480618 15108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260926 20:47:14.484256 15122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:14.485695 15108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:47:14.485884 15108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "5b64d2ab12784f30baa99c49544ef7f2"
format_stamp: "Formatted at 2026-09-26 20:47:14 on dist-test-slave-z7j8"
server_key: "6291257fb2a0eaedf24d5c279baa678d"
server_key_iv: "e3acdb528af1ff17029daa7a11498ac1"
server_key_version: "encryptionkey@0"
I20260926 20:47:14.486123 15108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:14.523633 15108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:14.524567 15108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:14.524854 15108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:14.525804 15108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:14.527366 15108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:14.527489 15108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:14.527637 15108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:14.527727 15108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:14.561177 15108 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:39537
I20260926 20:47:14.561226 15234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:39537 every 8 connection(s)
I20260926 20:47:14.562525 15108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:47:14.570881 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15108
I20260926 20:47:14.571092 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:47:14.571554 5432 external_mini_cluster.cc:1428] Setting key 48bb0f55988ac0c7d867760db1804da7
I20260926 20:47:14.576562 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.565293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:54199 (local address 127.5.78.62:39191)
0926 20:47:14.565497 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:14.565502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:14.567005 (+ 1503us) server_negotiation.cc:407] Connection header received
0926 20:47:14.568995 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:14.569002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:14.569108 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:14.569234 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:14.570479 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.571430 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.573221 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.573701 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.573871 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.575562 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:14.575568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:14.575572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:14.575593 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:14.575627 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:14.575641 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:14.575748 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:14.576019 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 20:47:14.576132 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:47:14.577733 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.565807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39191 (local address 127.5.78.3:54199)
0926 20:47:14.566823 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:14.566869 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:14.568661 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:14.569494 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:14.569504 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:14.569615 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:14.570240 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:14.570258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.571624 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.571629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:14.573032 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:14.573046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.573999 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.574009 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:14.574272 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.575162 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:47:14.575165 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:14.575211 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:47:14.575312 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:14.575752 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:14.575769 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:14.575781 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:47:14.576011 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 20:47:14.576503 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":278,"threads_started":1}
I20260926 20:47:14.579393 15235 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39191
I20260926 20:47:14.579770 15235 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:14.580619 15235 heartbeater.cc:511] Master 127.5.78.62:39191 requested a full tablet report, sending...
I20260926 20:47:14.582289 14788 ts_manager.cc:194] Registered new tserver with Master: 5b64d2ab12784f30baa99c49544ef7f2 (127.5.78.3:39537)
I20260926 20:47:14.583216 14788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.3:54199
I20260926 20:47:14.587631 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 20:47:14.594679 14976 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:14.592906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.5.78.62:39191)
0926 20:47:14.593100 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:14.593105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:14.593566 (+ 461us) server_negotiation.cc:407] Connection header received
0926 20:47:14.593753 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:14.593757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:14.593842 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:14.593939 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:14.594335 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47052: BlockingRecv error: recv got EOF from 127.0.0.1:47052 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:47:14.595534 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14847
I20260926 20:47:14.604296 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.596642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.5.78.1:41951)
0926 20:47:14.597315 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:14.597323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:14.597349 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:47:14.597410 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:14.597416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:14.597644 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:14.597827 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:14.598281 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.599369 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.600687 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.601322 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.601474 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.602745 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:14.602754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:14.602769 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:14.603388 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:14.603440 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:14.603469 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:14.603614 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:14.603777 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 20:47:14.603924 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":393,"threads_started":1}
I20260926 20:47:14.605700 14969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:14.687323 14974 heartbeater.cc:503] Master 127.5.78.62:39191 was elected leader, sending a full tablet report...
I20260926 20:47:14.688539 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14847
W20260926 20:47:14.705127 14773 connection.cc:535] server connection from 127.5.78.1:60289 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 20:47:14.705436 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14978
I20260926 20:47:14.713903 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.706355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.5.78.2:42305)
0926 20:47:14.706969 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:14.706977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:14.707005 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:47:14.707072 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:14.707079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:14.707301 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:14.707595 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:14.708014 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.709160 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.710651 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.711259 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.711370 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.712466 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:14.712476 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:14.712491 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:14.713129 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:14.713179 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:14.713204 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:14.713297 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:14.713477 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 20:47:14.713607 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":341,"threads_started":1}
I20260926 20:47:14.715372 15100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:14.797698 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14978
I20260926 20:47:14.813496 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15108
I20260926 20:47:14.821455 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:14.814457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.5.78.3:39537)
0926 20:47:14.815055 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:14.815062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:14.815091 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:47:14.815147 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:14.815154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:14.815367 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:14.815505 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:14.815931 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.816982 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.818195 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:14.818910 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:14.819010 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:14.819980 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:14.819989 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:14.820002 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:14.820603 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:14.820648 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:14.820667 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:14.820755 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:14.820939 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 20:47:14.821044 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":291,"threads_started":1}
I20260926 20:47:14.822798 15230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:14.903820 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15108
I20260926 20:47:14.921983 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 14757
I20260926 20:47:14.923239 14818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:15.023241 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 14757
2026-09-26T20:47:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2486 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T20:47:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:47:15Z Disabled control of system clock
I20260926 20:47:15.099582 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:40507
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:40507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 20:47:15.428618 15254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:15.428946 15254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:15.429095 15254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:15.436594 15254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:47:15.436724 15254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:15.436802 15254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:15.436896 15254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:47:15.436985 15254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:47:15.446358 15254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:40507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:40507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:15.447753 15254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:15.449595 15254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:15.462924 15259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:15.462867 15262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:15.462888 15260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:15.463723 15254 server_base.cc:1034] running on GCE node
I20260926 20:47:15.464705 15254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:15.466393 15254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:15.467717 15254 hybrid_clock.cc:648] HybridClock initialized: now 1790455635467651 us; error 75 us; skew 500 ppm
I20260926 20:47:15.468158 15254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:15.470927 15254 webserver.cc:460] Webserver started at http://127.5.78.62:43145/ using document root <none> and password file <none>
I20260926 20:47:15.471755 15254 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:15.471860 15254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:15.472275 15254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:15.475473 15254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "9643fef8478948aebeb1ae8e1a637cd9"
format_stamp: "Formatted at 2026-09-26 20:47:15 on dist-test-slave-z7j8"
server_key: "f912c34eca7350549810f9f9eb4b11ef"
server_key_iv: "157ccc87e1e7e2d8e49e1eaca3d14694"
server_key_version: "encryptionkey@0"
I20260926 20:47:15.476380 15254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "9643fef8478948aebeb1ae8e1a637cd9"
format_stamp: "Formatted at 2026-09-26 20:47:15 on dist-test-slave-z7j8"
server_key: "f912c34eca7350549810f9f9eb4b11ef"
server_key_iv: "157ccc87e1e7e2d8e49e1eaca3d14694"
server_key_version: "encryptionkey@0"
I20260926 20:47:15.482201 15254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:47:15.486111 15268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:15.487905 15254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:47:15.488128 15254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "9643fef8478948aebeb1ae8e1a637cd9"
format_stamp: "Formatted at 2026-09-26 20:47:15 on dist-test-slave-z7j8"
server_key: "f912c34eca7350549810f9f9eb4b11ef"
server_key_iv: "157ccc87e1e7e2d8e49e1eaca3d14694"
server_key_version: "encryptionkey@0"
I20260926 20:47:15.488399 15254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:15.495087 15254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:15.495952 15254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:15.496268 15254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:15.516289 15254 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:40507
I20260926 20:47:15.516330 15319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:40507 every 8 connection(s)
I20260926 20:47:15.517838 15254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:47:15.521674 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15254
I20260926 20:47:15.521931 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:47:15.522023 15320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:47:15.522699 5432 external_mini_cluster.cc:1428] Setting key d338e964e0597a7eb23ad3d3c1613bc5
I20260926 20:47:15.532573 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:15.524710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.5.78.62:40507)
0926 20:47:15.526020 (+ 1310us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:15.526047 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:15.526106 (+ 59us) server_negotiation.cc:407] Connection header received
0926 20:47:15.528644 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:15.528668 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:15.529133 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:15.529599 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:15.530837 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:15.530843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:15.530857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:15.530870 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:15.530924 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:15.530939 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:15.531210 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:15.531343 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 20:47:15.531612 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":441,"threads_started":1}
I20260926 20:47:15.535553 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9: Bootstrap starting.
I20260926 20:47:15.538918 15320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9: Neither blocks nor log segments found. Creating new log.
I20260926 20:47:15.540032 15320 log.cc:824] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:47:15.543334 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9: No bootstrap required, opened a new log
I20260926 20:47:15.551362 15320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9643fef8478948aebeb1ae8e1a637cd9" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40507 } }
I20260926 20:47:15.551865 15320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:47:15.551982 15320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9643fef8478948aebeb1ae8e1a637cd9, State: Initialized, Role: FOLLOWER
I20260926 20:47:15.552765 15320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9643fef8478948aebeb1ae8e1a637cd9" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40507 } }
I20260926 20:47:15.553062 15320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:47:15.553232 15320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:47:15.553432 15320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:47:15.555824 15320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9643fef8478948aebeb1ae8e1a637cd9" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40507 } }
I20260926 20:47:15.556416 15320 leader_election.cc:302] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9643fef8478948aebeb1ae8e1a637cd9; no voters:
I20260926 20:47:15.557063 15320 leader_election.cc:288] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:47:15.557380 15325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:47:15.558276 15325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [term 1 LEADER]: Becoming Leader. State: Replica: 9643fef8478948aebeb1ae8e1a637cd9, State: Running, Role: LEADER
I20260926 20:47:15.558995 15325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9643fef8478948aebeb1ae8e1a637cd9" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40507 } }
I20260926 20:47:15.559429 15320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:47:15.562379 15327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9643fef8478948aebeb1ae8e1a637cd9. Latest consensus state: current_term: 1 leader_uuid: "9643fef8478948aebeb1ae8e1a637cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9643fef8478948aebeb1ae8e1a637cd9" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40507 } } }
I20260926 20:47:15.562516 15326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9643fef8478948aebeb1ae8e1a637cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9643fef8478948aebeb1ae8e1a637cd9" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40507 } } }
I20260926 20:47:15.562861 15327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:47:15.562963 15326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:47:15.565327 15331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:47:15.571703 15331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:47:15.581547 15331 catalog_manager.cc:1347] Generated new cluster ID: 22672371bdab4c73921296c8084fb596
I20260926 20:47:15.581693 15331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:47:15.602198 15331 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:47:15.602727 15331 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:47:15.610690 15331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9643fef8478948aebeb1ae8e1a637cd9: Generated new TSK 0
I20260926 20:47:15.611724 15331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:47:15.617807 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:40507
--builtin_ntp_servers=127.5.78.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 20:47:15.961400 15344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:15.961787 15344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:15.961956 15344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:15.969527 15344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:15.969638 15344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:15.969822 15344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:47:15.979342 15344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:40507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:15.980705 15344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:15.982698 15344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:15.996881 15349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:15.996940 15350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:15.996960 15352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:16.003297 15344 server_base.cc:1034] running on GCE node
I20260926 20:47:16.004041 15344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:16.005107 15344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:16.006352 15344 hybrid_clock.cc:648] HybridClock initialized: now 1790455636006308 us; error 52 us; skew 500 ppm
I20260926 20:47:16.006721 15344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:16.009187 15344 webserver.cc:460] Webserver started at http://127.5.78.1:33339/ using document root <none> and password file <none>
I20260926 20:47:16.009922 15344 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:16.010017 15344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:16.010298 15344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:16.015316 15344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "eb8ded315e024e7e9be27f3237510b00"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "050af444e697e0bab56415fd94c54b89"
server_key_iv: "28789681464d6ead53b5e0aa2748e46f"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.016074 15344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "eb8ded315e024e7e9be27f3237510b00"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "050af444e697e0bab56415fd94c54b89"
server_key_iv: "28789681464d6ead53b5e0aa2748e46f"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.021525 15344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:47:16.025126 15358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.026690 15344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:47:16.026842 15344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "eb8ded315e024e7e9be27f3237510b00"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "050af444e697e0bab56415fd94c54b89"
server_key_iv: "28789681464d6ead53b5e0aa2748e46f"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.027031 15344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:16.032784 15344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:16.033572 15344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:16.033797 15344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:16.034896 15344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:16.036448 15344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:16.036541 15344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.036619 15344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:16.036676 15344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.071774 15344 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:39283
I20260926 20:47:16.071821 15470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:39283 every 8 connection(s)
I20260926 20:47:16.073221 15344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:47:16.074395 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15344
I20260926 20:47:16.074641 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:47:16.075426 5432 external_mini_cluster.cc:1428] Setting key 2f20de6eccbdca909f4e3fd7beef61a3
I20260926 20:47:16.078779 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:40507
--builtin_ntp_servers=127.5.78.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 20:47:16.083886 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.076280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:52757 (local address 127.5.78.62:40507)
0926 20:47:16.076725 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:16.076731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:16.078292 (+ 1561us) server_negotiation.cc:407] Connection header received
0926 20:47:16.081182 (+ 2890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:16.081187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:16.081262 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:16.081379 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:16.083097 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:16.083102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:16.083105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:16.083123 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:16.083167 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:16.083179 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:16.083310 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:16.083538 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 20:47:16.083590 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":194,"threads_started":1}
I20260926 20:47:16.085225 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.077007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:40507 (local address 127.5.78.1:52757)
0926 20:47:16.078118 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:16.078165 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:16.080793 (+ 2628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:16.081688 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:16.081703 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:16.081855 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:16.082704 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:47:16.082710 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:16.082776 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:47:16.082893 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:16.083307 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:16.083321 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:16.083331 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 20:47:16.083625 (+ 294us) client_negotiation.cc:240] Negotiation successful
0926 20:47:16.083939 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":309,"threads_started":1}
I20260926 20:47:16.086884 15471 heartbeater.cc:347] Connected to a master server at 127.5.78.62:40507
I20260926 20:47:16.087217 15471 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:16.087831 15471 heartbeater.cc:511] Master 127.5.78.62:40507 requested a full tablet report, sending...
I20260926 20:47:16.090039 15285 ts_manager.cc:194] Registered new tserver with Master: eb8ded315e024e7e9be27f3237510b00 (127.5.78.1:39283)
W20260926 20:47:16.413231 15475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:16.413522 15475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:16.413609 15475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:16.420795 15475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:16.420920 15475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:16.421030 15475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:47:16.430213 15475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:40507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:16.431653 15475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:16.433450 15475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:16.446875 15481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:16.446925 15480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:16.446836 15483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:16.451786 15475 server_base.cc:1034] running on GCE node
I20260926 20:47:16.452440 15475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:16.453420 15475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:16.454656 15475 hybrid_clock.cc:648] HybridClock initialized: now 1790455636454625 us; error 50 us; skew 500 ppm
I20260926 20:47:16.455003 15475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:16.457381 15475 webserver.cc:460] Webserver started at http://127.5.78.2:34641/ using document root <none> and password file <none>
I20260926 20:47:16.458145 15475 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:16.458295 15475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:16.458693 15475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:16.461692 15475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "20177d1ecfcb4657bc3969dc19f72620"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "fa304ae7b6bd7130b104e9957529096b"
server_key_iv: "0bdb22458bd28b184a217e83eaeffdc4"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.462594 15475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "20177d1ecfcb4657bc3969dc19f72620"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "fa304ae7b6bd7130b104e9957529096b"
server_key_iv: "0bdb22458bd28b184a217e83eaeffdc4"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.468122 15475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 20:47:16.471899 15489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.473471 15475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:47:16.473666 15475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "20177d1ecfcb4657bc3969dc19f72620"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "fa304ae7b6bd7130b104e9957529096b"
server_key_iv: "0bdb22458bd28b184a217e83eaeffdc4"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.473899 15475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:16.479949 15475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:16.480825 15475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:16.481107 15475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:16.482069 15475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:16.483623 15475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:16.483760 15475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.483894 15475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:16.483983 15475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.519414 15475 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:40247
I20260926 20:47:16.519433 15601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:40247 every 8 connection(s)
I20260926 20:47:16.520764 15475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:47:16.525195 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15475
I20260926 20:47:16.525398 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:47:16.525915 5432 external_mini_cluster.cc:1428] Setting key d01a60cd9c975b1a9b2ec3bf5f032341
I20260926 20:47:16.528932 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:40507
--builtin_ntp_servers=127.5.78.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 20:47:16.533708 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.523636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:40901 (local address 127.5.78.62:40507)
0926 20:47:16.523800 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:16.523805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:16.525558 (+ 1753us) server_negotiation.cc:407] Connection header received
0926 20:47:16.527631 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:16.527636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:16.527728 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:16.527833 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:16.532769 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:16.532776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:16.532783 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:16.532810 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:16.532853 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:16.532872 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:16.533006 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:16.533254 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 20:47:16.533333 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:47:16.535003 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.524264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:40507 (local address 127.5.78.2:40901)
0926 20:47:16.525361 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:16.525425 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:16.527364 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:16.528108 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:16.528121 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:16.528261 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:16.532340 (+ 4079us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:47:16.532344 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:16.532401 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:47:16.532526 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:16.532989 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:16.533000 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:16.533008 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 20:47:16.533321 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 20:47:16.533611 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":275,"threads_started":1}
I20260926 20:47:16.536675 15602 heartbeater.cc:347] Connected to a master server at 127.5.78.62:40507
I20260926 20:47:16.537010 15602 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:16.537628 15602 heartbeater.cc:511] Master 127.5.78.62:40507 requested a full tablet report, sending...
I20260926 20:47:16.539130 15285 ts_manager.cc:194] Registered new tserver with Master: 20177d1ecfcb4657bc3969dc19f72620 (127.5.78.2:40247)
W20260926 20:47:16.868130 15605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:16.868500 15605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:16.868651 15605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:16.876199 15605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:16.876363 15605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:16.876543 15605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:47:16.886031 15605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:40507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:16.887409 15605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:16.889222 15605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:16.903059 15610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:16.903031 15613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:16.902988 15611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:16.904559 15605 server_base.cc:1034] running on GCE node
I20260926 20:47:16.905184 15605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:16.906133 15605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:16.907363 15605 hybrid_clock.cc:648] HybridClock initialized: now 1790455636907299 us; error 52 us; skew 500 ppm
I20260926 20:47:16.907761 15605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:47:16.910184 15605 webserver.cc:460] Webserver started at http://127.5.78.3:44521/ using document root <none> and password file <none>
I20260926 20:47:16.910957 15605 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:16.911043 15605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:16.911403 15605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:16.914495 15605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "5317fdc30faa4e218705a1eb543c6b36"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "f04b92eafcbcdf42f550085e177b4145"
server_key_iv: "1cf9e0e22bdffac7ac452f5e886e2f2b"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.915424 15605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "5317fdc30faa4e218705a1eb543c6b36"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "f04b92eafcbcdf42f550085e177b4145"
server_key_iv: "1cf9e0e22bdffac7ac452f5e886e2f2b"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.921255 15605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:47:16.925379 15619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.926992 15605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:47:16.927213 15605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "5317fdc30faa4e218705a1eb543c6b36"
format_stamp: "Formatted at 2026-09-26 20:47:16 on dist-test-slave-z7j8"
server_key: "f04b92eafcbcdf42f550085e177b4145"
server_key_iv: "1cf9e0e22bdffac7ac452f5e886e2f2b"
server_key_version: "encryptionkey@0"
I20260926 20:47:16.927462 15605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:16.933580 15605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:16.934535 15605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:16.934832 15605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:16.935902 15605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:16.937422 15605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:16.937551 15605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.937681 15605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:16.937768 15605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:16.972237 15605 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:33195
I20260926 20:47:16.972293 15731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:33195 every 8 connection(s)
I20260926 20:47:16.973656 15605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:47:16.974071 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15605
I20260926 20:47:16.974269 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:47:16.974869 5432 external_mini_cluster.cc:1428] Setting key da61b8c0d696f568df7a22743d516b6f
I20260926 20:47:16.982506 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.976665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:40147 (local address 127.5.78.62:40507)
0926 20:47:16.976870 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:16.976876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:16.978529 (+ 1653us) server_negotiation.cc:407] Connection header received
0926 20:47:16.980334 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:16.980342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:16.980410 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:16.980503 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:16.981789 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:16.981794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:16.981797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:16.981814 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:16.981847 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:16.981859 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:16.981960 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:16.982131 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 20:47:16.982189 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:47:16.983479 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.977232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:40507 (local address 127.5.78.3:40147)
0926 20:47:16.978348 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:16.978431 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:16.980023 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:16.980727 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:16.980736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:16.980837 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:16.981479 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:47:16.981481 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:16.981520 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:47:16.981610 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:16.981950 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:16.981957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:16.981963 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 20:47:16.982180 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 20:47:16.982463 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":361,"threads_started":1}
I20260926 20:47:16.984968 15732 heartbeater.cc:347] Connected to a master server at 127.5.78.62:40507
I20260926 20:47:16.985330 15732 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:16.985985 15732 heartbeater.cc:511] Master 127.5.78.62:40507 requested a full tablet report, sending...
I20260926 20:47:16.987763 15285 ts_manager.cc:194] Registered new tserver with Master: 5317fdc30faa4e218705a1eb543c6b36 (127.5.78.3:33195)
I20260926 20:47:16.991106 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 20:47:16.998060 15473 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:47:16.996273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.5.78.62:40507)
0926 20:47:16.996469 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:16.996474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:16.997083 (+ 609us) server_negotiation.cc:407] Connection header received
0926 20:47:16.997217 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:16.997221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:16.997278 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:16.997379 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:16.997762 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40994: BlockingRecv error: recv got EOF from 127.0.0.1:40994 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:47:16.998879 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15344
I20260926 20:47:17.003032 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:16.999860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.5.78.1:39283)
0926 20:47:17.000415 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:17.000432 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:17.000459 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:47:17.000516 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:17.000521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:17.000752 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:17.000882 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:17.002029 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:17.002040 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:17.002062 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:17.002109 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:17.002188 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:17.002206 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:17.002325 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:17.002565 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 20:47:17.002700 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":268,"threads_started":1}
I20260926 20:47:17.004393 15466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:17.090099 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15344
I20260926 20:47:17.107704 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15475
I20260926 20:47:17.111568 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:17.108638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.5.78.2:40247)
0926 20:47:17.109146 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:17.109155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:17.109185 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:47:17.109248 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:17.109255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:17.109486 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:17.109707 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:17.110834 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:17.110840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:17.110853 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:17.110864 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:17.110908 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:17.110922 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:17.111080 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:17.111174 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 20:47:17.111284 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":242,"threads_started":1}
I20260926 20:47:17.112883 15597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:17.194197 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15475
I20260926 20:47:17.210609 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15605
I20260926 20:47:17.214710 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:17.211598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.5.78.3:33195)
0926 20:47:17.212274 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:17.212281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:17.212315 (+ 34us) server_negotiation.cc:407] Connection header received
0926 20:47:17.212380 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:17.212388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:17.212609 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:17.212759 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:17.213758 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:17.213767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:17.213779 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:47:17.213796 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:17.213838 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:47:17.213853 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:17.214003 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:17.214183 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 20:47:17.214315 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":316,"threads_started":1}
I20260926 20:47:17.216022 15727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:17.299120 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15605
I20260926 20:47:17.315255 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15254
I20260926 20:47:17.316442 15315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:17.415408 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15254
2026-09-26T20:47:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2392 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:47:17 dist-test-slave-z7j8 krb5kdc[15750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:47:17 dist-test-slave-z7j8 krb5kdc[15750](info): set up 2 sockets
Sep 26 20:47:17 dist-test-slave-z7j8 krb5kdc[15750](info): commencing operation
krb5kdc: starting...
W20260926 20:47:18.523480 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:47:18 dist-test-slave-z7j8 krb5kdc[15750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:47:18.763136 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:45671
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:45671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:47:19.104544 15763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:19.104843 15763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:19.104931 15763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:19.112404 15763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:47:19.112533 15763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:19.112581 15763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:19.112637 15763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:47:19.112681 15763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:47:19.122736 15763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:45671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:45671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:19.124060 15763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:19.125353 15763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:19.136989 15768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:19.136989 15771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:19.137048 15769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:19.137848 15763 server_base.cc:1034] running on GCE node
I20260926 20:47:19.138731 15763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:19.140232 15763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:19.141453 15763 hybrid_clock.cc:648] HybridClock initialized: now 1790455639141416 us; error 63 us; skew 500 ppm
I20260926 20:47:19.141794 15763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:47:19 dist-test-slave-z7j8 krb5kdc[15750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:47:19.144881 15763 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:47:19.146333 15763 webserver.cc:460] Webserver started at http://127.5.78.62:40919/ using document root <none> and password file <none>
I20260926 20:47:19.147109 15763 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:19.147212 15763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:19.147526 15763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:19.150584 15763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "ce8dcd7905ef4ff4bfd0939eed437721"
format_stamp: "Formatted at 2026-09-26 20:47:19 on dist-test-slave-z7j8"
server_key: "c21ec3f0b3499b259a2d2b8277b91267"
server_key_iv: "01b07eb2b644ec45fd057f01fc6e99b0"
server_key_version: "encryptionkey@0"
I20260926 20:47:19.151400 15763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "ce8dcd7905ef4ff4bfd0939eed437721"
format_stamp: "Formatted at 2026-09-26 20:47:19 on dist-test-slave-z7j8"
server_key: "c21ec3f0b3499b259a2d2b8277b91267"
server_key_iv: "01b07eb2b644ec45fd057f01fc6e99b0"
server_key_version: "encryptionkey@0"
I20260926 20:47:19.157092 15763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:47:19.160815 15778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:19.162352 15763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 20:47:19.162626 15763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ce8dcd7905ef4ff4bfd0939eed437721"
format_stamp: "Formatted at 2026-09-26 20:47:19 on dist-test-slave-z7j8"
server_key: "c21ec3f0b3499b259a2d2b8277b91267"
server_key_iv: "01b07eb2b644ec45fd057f01fc6e99b0"
server_key_version: "encryptionkey@0"
I20260926 20:47:19.162814 15763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:19.180637 15763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:19.184928 15763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:19.185235 15763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:19.204411 15763 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:45671
I20260926 20:47:19.204434 15829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:45671 every 8 connection(s)
I20260926 20:47:19.205809 15763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:47:19.206494 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15763
I20260926 20:47:19.206736 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:47:19.207574 5432 external_mini_cluster.cc:1428] Setting key e834e9da9963b10fb00701a85d93384d
I20260926 20:47:19.210242 15830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:47:19 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:47:19.223498 15830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721: Bootstrap starting.
I20260926 20:47:19.228390 15830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721: Neither blocks nor log segments found. Creating new log.
I20260926 20:47:19.229802 15830 log.cc:824] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721: Log is configured to *not* fsync() on all Append() calls
I20260926 20:47:19.233888 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:19.209482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.5.78.62:45671)
0926 20:47:19.210268 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:19.210279 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:19.210352 (+ 73us) server_negotiation.cc:407] Connection header received
0926 20:47:19.211983 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:19.212007 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:19.212406 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:19.212783 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:19.214604 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:19.215999 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:19.217792 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:19.218566 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:19.218693 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:19.223459 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:19.223472 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:19.223500 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:19.223564 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:19.228579 (+ 5015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:19.229621 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:19.229630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:19.229639 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:19.229976 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:19.230575 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:19.230582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:19.230587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:19.231344 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:19.231662 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:19.232310 (+ 648us) server_negotiation.cc:299] Negotiation successful
0926 20:47:19.232804 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":363,"threads_started":1}
I20260926 20:47:19.237607 15830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721: No bootstrap required, opened a new log
I20260926 20:47:19.247084 15830 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45671 } }
I20260926 20:47:19.247534 15830 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:47:19.247644 15830 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8dcd7905ef4ff4bfd0939eed437721, State: Initialized, Role: FOLLOWER
I20260926 20:47:19.248272 15830 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45671 } }
I20260926 20:47:19.248525 15830 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:47:19.248670 15830 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:47:19.248843 15830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:47:19.251399 15830 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45671 } }
I20260926 20:47:19.252023 15830 leader_election.cc:302] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce8dcd7905ef4ff4bfd0939eed437721; no voters:
I20260926 20:47:19.252729 15830 leader_election.cc:288] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:47:19.252866 15835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:47:19.253997 15835 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [term 1 LEADER]: Becoming Leader. State: Replica: ce8dcd7905ef4ff4bfd0939eed437721, State: Running, Role: LEADER
I20260926 20:47:19.255045 15835 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45671 } }
I20260926 20:47:19.255234 15830 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:47:19.257815 15836 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45671 } } }
I20260926 20:47:19.258210 15836 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [sys.catalog]: This master's current role is: LEADER
I20260926 20:47:19.258695 15837 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce8dcd7905ef4ff4bfd0939eed437721. Latest consensus state: current_term: 1 leader_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8dcd7905ef4ff4bfd0939eed437721" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 45671 } } }
I20260926 20:47:19.259021 15837 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721 [sys.catalog]: This master's current role is: LEADER
I20260926 20:47:19.259196 15842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:47:19.266115 15842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:47:19.281196 15842 catalog_manager.cc:1347] Generated new cluster ID: 7db4a20391af4361aa344e5ed991e651
I20260926 20:47:19.281329 15842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:47:19.309332 15842 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:47:19.311265 15842 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:47:19.330210 15842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721: Generated new TSK 0
I20260926 20:47:19.331427 15842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:47:19.402138 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45671
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:47:19.748183 15856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:19.748473 15856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:19.748607 15856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:19.756312 15856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:19.756469 15856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:19.756654 15856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:47:19.766867 15856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:19.768445 15856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:19.769933 15856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:19.783828 15861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:19.783792 15864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:19.783864 15862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:19.785140 15856 server_base.cc:1034] running on GCE node
I20260926 20:47:19.785841 15856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:19.786921 15856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:19.788166 15856 hybrid_clock.cc:648] HybridClock initialized: now 1790455639788113 us; error 47 us; skew 500 ppm
I20260926 20:47:19.788529 15856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:47:19 dist-test-slave-z7j8 krb5kdc[15750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:47:19.791852 15856 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:47:19.793437 15856 webserver.cc:460] Webserver started at http://127.5.78.1:41031/ using document root <none> and password file <none>
I20260926 20:47:19.794222 15856 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:19.794379 15856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:19.794723 15856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:19.797735 15856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "4ee2a3888c1e492b822eb7f3a200a0d9"
format_stamp: "Formatted at 2026-09-26 20:47:19 on dist-test-slave-z7j8"
server_key: "e84f42cb03d4faed2d9a851682856a1f"
server_key_iv: "e8e7fdf01f1c5ba85a62ce15e19d35d5"
server_key_version: "encryptionkey@0"
I20260926 20:47:19.798687 15856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "4ee2a3888c1e492b822eb7f3a200a0d9"
format_stamp: "Formatted at 2026-09-26 20:47:19 on dist-test-slave-z7j8"
server_key: "e84f42cb03d4faed2d9a851682856a1f"
server_key_iv: "e8e7fdf01f1c5ba85a62ce15e19d35d5"
server_key_version: "encryptionkey@0"
I20260926 20:47:19.804735 15856 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 20:47:19.808987 15871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:19.810758 15856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:47:19.810974 15856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "4ee2a3888c1e492b822eb7f3a200a0d9"
format_stamp: "Formatted at 2026-09-26 20:47:19 on dist-test-slave-z7j8"
server_key: "e84f42cb03d4faed2d9a851682856a1f"
server_key_iv: "e8e7fdf01f1c5ba85a62ce15e19d35d5"
server_key_version: "encryptionkey@0"
I20260926 20:47:19.811215 15856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:19.827900 15856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:19.832381 15856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:19.832703 15856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:19.833684 15856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:19.835108 15856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:19.835210 15856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:19.835368 15856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:19.835443 15856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:19.869726 15856 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:44745
I20260926 20:47:19.869788 15983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:44745 every 8 connection(s)
I20260926 20:47:19.871120 15856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:47:19.876047 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15856
I20260926 20:47:19.876264 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:47:19.876858 5432 external_mini_cluster.cc:1428] Setting key c26568e129fed0c707b0af3ca8af4035
Sep 26 20:47:19 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:47:19.892398 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:19.873686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:53893 (local address 127.5.78.62:45671)
0926 20:47:19.874101 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:19.874106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:19.875231 (+ 1125us) server_negotiation.cc:407] Connection header received
0926 20:47:19.877235 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:19.877241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:19.877332 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:19.877461 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:19.879404 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:19.880161 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:19.881836 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:19.882246 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:19.882396 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:19.886141 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:19.886147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:19.886150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:19.886183 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:19.888786 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:19.889819 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:19.889824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:19.889826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:19.889953 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:19.890552 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:19.890557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:19.890559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:19.890845 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:19.890988 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:19.891935 (+ 947us) server_negotiation.cc:299] Negotiation successful
0926 20:47:19.892127 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":207,"threads_started":1}
I20260926 20:47:19.893961 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:19.874259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:45671 (local address 127.5.78.1:53893)
0926 20:47:19.875058 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:19.875111 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:19.876927 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:19.877739 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:19.877748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:19.878589 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:19.879215 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:19.879234 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:19.880354 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:19.880358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:19.881668 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:19.881681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:19.882542 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:19.882550 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:19.882792 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:19.883534 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:19.883558 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:19.885834 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:19.888947 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:47:19.888960 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:47:19.888983 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:47:19.889658 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:47:19.890068 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:47:19.890076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:47:19.890082 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:47:19.890372 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:47:19.890992 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:19.891003 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:19.891513 (+ 510us) client_negotiation.cc:769] Sending connection context
0926 20:47:19.891837 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 20:47:19.892469 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":348,"threads_started":1}
I20260926 20:47:19.895742 15984 heartbeater.cc:347] Connected to a master server at 127.5.78.62:45671
I20260926 20:47:19.896081 15984 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:19.896930 15984 heartbeater.cc:511] Master 127.5.78.62:45671 requested a full tablet report, sending...
I20260926 20:47:19.899331 15795 ts_manager.cc:194] Registered new tserver with Master: 4ee2a3888c1e492b822eb7f3a200a0d9 (127.5.78.1:44745)
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:47:19.900849 15795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:53893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:47:19.934346 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45671
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:47:20.277043 15990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:20.277354 15990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:20.277514 15990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:20.284870 15990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:20.284993 15990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:20.285176 15990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:47:20.295020 15990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:20.296708 15990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:20.298449 15990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:20.314700 15995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:20.314771 15996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:20.314796 15998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:20.316009 15990 server_base.cc:1034] running on GCE node
I20260926 20:47:20.316780 15990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:20.317840 15990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:20.319041 15990 hybrid_clock.cc:648] HybridClock initialized: now 1790455640318978 us; error 75 us; skew 500 ppm
I20260926 20:47:20.319396 15990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:47:20 dist-test-slave-z7j8 krb5kdc[15750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455640, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:47:20.322738 15990 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:47:20.324209 15990 webserver.cc:460] Webserver started at http://127.5.78.2:40339/ using document root <none> and password file <none>
I20260926 20:47:20.324906 15990 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:20.325009 15990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:20.325270 15990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:20.328346 15990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "4dfb58f47da9485095c430ca76f83c99"
format_stamp: "Formatted at 2026-09-26 20:47:20 on dist-test-slave-z7j8"
server_key: "b807fa984b1f6a7f951410932e47e696"
server_key_iv: "2891bccd4ad183d93047d73df6a32914"
server_key_version: "encryptionkey@0"
I20260926 20:47:20.329128 15990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "4dfb58f47da9485095c430ca76f83c99"
format_stamp: "Formatted at 2026-09-26 20:47:20 on dist-test-slave-z7j8"
server_key: "b807fa984b1f6a7f951410932e47e696"
server_key_iv: "2891bccd4ad183d93047d73df6a32914"
server_key_version: "encryptionkey@0"
I20260926 20:47:20.334864 15990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:47:20.338737 16005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:20.340113 15990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:47:20.340303 15990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "4dfb58f47da9485095c430ca76f83c99"
format_stamp: "Formatted at 2026-09-26 20:47:20 on dist-test-slave-z7j8"
server_key: "b807fa984b1f6a7f951410932e47e696"
server_key_iv: "2891bccd4ad183d93047d73df6a32914"
server_key_version: "encryptionkey@0"
I20260926 20:47:20.340512 15990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:20.371526 15990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:20.376360 15990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:20.376713 15990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:20.377840 15990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:20.379446 15990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:20.379572 15990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:20.379745 15990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:20.379837 15990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:20.416662 15990 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:36455
I20260926 20:47:20.416672 16117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:36455 every 8 connection(s)
I20260926 20:47:20.418125 15990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:47:20.418692 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 15990
I20260926 20:47:20.418900 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:47:20.419437 5432 external_mini_cluster.cc:1428] Setting key 922dd0b261354055bf3e3ab9046dccbc
Sep 26 20:47:20 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455640, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:47:20.440164 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:20.421219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:47571 (local address 127.5.78.62:45671)
0926 20:47:20.421672 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:20.421683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:20.423056 (+ 1373us) server_negotiation.cc:407] Connection header received
0926 20:47:20.424918 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:20.424923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:20.425022 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:20.425145 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:20.426836 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.427532 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.429113 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.429530 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.429674 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:20.433739 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:20.433745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:20.433748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:20.433786 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:20.436671 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:20.437602 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:20.437607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:20.437609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:20.437755 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:20.438243 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:20.438248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:20.438250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:20.438587 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:20.438748 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:20.439544 (+ 796us) server_negotiation.cc:299] Negotiation successful
0926 20:47:20.439790 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":219,"threads_started":1}
I20260926 20:47:20.441170 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:20.421985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:45671 (local address 127.5.78.2:47571)
0926 20:47:20.422871 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:20.422929 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:20.424612 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:20.425418 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:20.425427 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:20.425972 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:20.426657 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:20.426677 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.427697 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.427707 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:20.428956 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:20.428968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.429776 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.429783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:20.430024 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:20.430832 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:20.430857 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:20.433443 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:20.436893 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:47:20.436905 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:47:20.436928 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:47:20.437433 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:47:20.437884 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:47:20.437890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:47:20.437894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:47:20.438108 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:47:20.438739 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:20.438748 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:20.439146 (+ 398us) client_negotiation.cc:769] Sending connection context
0926 20:47:20.439441 (+ 295us) client_negotiation.cc:240] Negotiation successful
0926 20:47:20.440031 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":408,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:47:20.442761 16118 heartbeater.cc:347] Connected to a master server at 127.5.78.62:45671
I20260926 20:47:20.443078 16118 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:20.443984 16118 heartbeater.cc:511] Master 127.5.78.62:45671 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:47:20.445940 15795 ts_manager.cc:194] Registered new tserver with Master: 4dfb58f47da9485095c430ca76f83c99 (127.5.78.2:36455)
I20260926 20:47:20.447314 15795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:47571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:47:20.475868 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45671
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:47:20.818820 16124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:47:20.819118 16124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:47:20.819216 16124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:47:20.826560 16124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:47:20.826696 16124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:47:20.826812 16124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:47:20.836894 16124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:45671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:47:20.838189 16124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:47:20.839656 16124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:47:20.854130 16130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:47:20.854172 16129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:20.854378 16124 server_base.cc:1034] running on GCE node
W20260926 20:47:20.854085 16132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:47:20.855305 16124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:47:20.856297 16124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:47:20.857568 16124 hybrid_clock.cc:648] HybridClock initialized: now 1790455640857514 us; error 33 us; skew 500 ppm
I20260926 20:47:20.857900 16124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:47:20 dist-test-slave-z7j8 krb5kdc[15750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455640, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:47:20.861157 16124 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:47:20.862892 16124 webserver.cc:460] Webserver started at http://127.5.78.3:39155/ using document root <none> and password file <none>
I20260926 20:47:20.863624 16124 fs_manager.cc:359] Metadata directory not provided
I20260926 20:47:20.863765 16124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:47:20.864090 16124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:47:20.867257 16124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "930b8a3f1c36420ab0dff42f96faa814"
format_stamp: "Formatted at 2026-09-26 20:47:20 on dist-test-slave-z7j8"
server_key: "8ccca6aadadbf6d468389e59e7cf3ce6"
server_key_iv: "83dfac15cdfc2aac98181befc7b62444"
server_key_version: "encryptionkey@0"
I20260926 20:47:20.868088 16124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "930b8a3f1c36420ab0dff42f96faa814"
format_stamp: "Formatted at 2026-09-26 20:47:20 on dist-test-slave-z7j8"
server_key: "8ccca6aadadbf6d468389e59e7cf3ce6"
server_key_iv: "83dfac15cdfc2aac98181befc7b62444"
server_key_version: "encryptionkey@0"
I20260926 20:47:20.873726 16124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:47:20.877450 16139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:20.879112 16124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 20:47:20.879338 16124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "930b8a3f1c36420ab0dff42f96faa814"
format_stamp: "Formatted at 2026-09-26 20:47:20 on dist-test-slave-z7j8"
server_key: "8ccca6aadadbf6d468389e59e7cf3ce6"
server_key_iv: "83dfac15cdfc2aac98181befc7b62444"
server_key_version: "encryptionkey@0"
I20260926 20:47:20.879561 16124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:47:20.893332 16124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:47:20.898129 16124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:47:20.898504 16124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:47:20.899521 16124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:47:20.900880 16124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:47:20.900985 16124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:20.901134 16124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:47:20.901208 16124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:47:20.904228 15984 heartbeater.cc:503] Master 127.5.78.62:45671 was elected leader, sending a full tablet report...
I20260926 20:47:20.936305 16124 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:37661
I20260926 20:47:20.936331 16251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:37661 every 8 connection(s)
I20260926 20:47:20.938167 16124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:47:20.940871 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 16124
I20260926 20:47:20.941071 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:47:20.941582 5432 external_mini_cluster.cc:1428] Setting key a6e68c80f0f1dcfe4212b473cde516cc
Sep 26 20:47:20 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455640, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:47:20.961802 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:20.941188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:42821 (local address 127.5.78.62:45671)
0926 20:47:20.941626 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:20.941651 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:20.943057 (+ 1406us) server_negotiation.cc:407] Connection header received
0926 20:47:20.945338 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:20.945343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:20.945421 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:20.945519 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:20.947340 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.948147 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.949996 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.950522 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.950642 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:20.954379 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:20.954385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:20.954388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:20.954487 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:20.957924 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:20.959035 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:20.959040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:20.959042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:20.959183 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:20.960001 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:20.960005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:20.960008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:20.960298 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:20.960439 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:20.961308 (+ 869us) server_negotiation.cc:299] Negotiation successful
0926 20:47:20.961504 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":233,"threads_started":1}
I20260926 20:47:20.963012 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:20.941930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:45671 (local address 127.5.78.3:42821)
0926 20:47:20.942889 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:20.942940 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:20.945034 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:20.945847 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:20.945857 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:20.946497 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:47:20.947135 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:20.947155 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.948315 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.948319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:20.949803 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:20.949818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.950744 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.950750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:20.950991 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:20.951669 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:47:20.951693 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:47:20.954082 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:47:20.958125 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:47:20.958134 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:47:20.958151 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:47:20.958886 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:47:20.959484 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:47:20.959493 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:47:20.959498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:47:20.959814 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:47:20.960463 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:47:20.960470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:47:20.960872 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 20:47:20.961137 (+ 265us) client_negotiation.cc:240] Negotiation successful
0926 20:47:20.961646 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":418,"threads_started":1}
I20260926 20:47:20.965001 16252 heartbeater.cc:347] Connected to a master server at 127.5.78.62:45671
I20260926 20:47:20.965345 16252 heartbeater.cc:464] Registering TS with master...
I20260926 20:47:20.966113 16252 heartbeater.cc:511] Master 127.5.78.62:45671 requested a full tablet report, sending...
I20260926 20:47:20.967897 15795 ts_manager.cc:194] Registered new tserver with Master: 930b8a3f1c36420ab0dff42f96faa814 (127.5.78.3:37661)
I20260926 20:47:20.969013 15795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:42821
I20260926 20:47:20.970461 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:47:20.989060 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:20.975261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.5.78.62:45671)
0926 20:47:20.975498 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:20.975508 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:20.976000 (+ 492us) server_negotiation.cc:407] Connection header received
0926 20:47:20.976228 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:20.976232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:20.976302 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:20.976407 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:20.978023 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.978908 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.980275 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:20.980864 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:20.980994 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:20.983510 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:20.983516 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:20.983528 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:20.983567 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:20.986377 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:20.987193 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:20.987197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:20.987200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:20.987346 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:20.987798 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:20.987802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:20.987804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:20.988134 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:20.988352 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:20.988655 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 20:47:20.988840 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:47:20.993603 15795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:47:21.000777 15795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:47:21.058116 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.024119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.5.78.2:36455)
0926 20:47:21.044400 (+ 20281us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.044413 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.048111 (+ 3698us) server_negotiation.cc:407] Connection header received
0926 20:47:21.048307 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.048315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.048657 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.048884 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:47:21.049678 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.051474 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.055117 (+ 3643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.056546 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.056653 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.056820 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.057638 (+ 818us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.057789 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20043,"thread_start_us":20288,"threads_started":1}
I20260926 20:47:21.058122 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.019686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.5.78.1:44745)
0926 20:47:21.046999 (+ 27313us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.047012 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.047053 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:47:21.047160 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.047171 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.047479 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.047679 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:47:21.049220 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.051736 (+ 2516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.054754 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.056336 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.056438 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.056768 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.057611 (+ 843us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.057789 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27063,"thread_start_us":27203,"threads_started":1}
I20260926 20:47:21.058159 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.019657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:37661 (local address 127.0.0.1:42638)
0926 20:47:21.047530 (+ 27873us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:21.047552 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:21.047732 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:21.048724 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:21.048733 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:21.048765 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:47:21.049299 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.049312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.051717 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.051724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.054380 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.054395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.056916 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.056923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.057360 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.057403 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 20:47:21.057494 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 20:47:21.057682 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27734,"mutex_wait_us":126,"thread_start_us":29196,"threads_started":1}
I20260926 20:47:21.059113 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.019195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:36455 (local address 127.0.0.1:60850)
0926 20:47:21.047984 (+ 28789us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:21.048001 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:21.048137 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:21.048887 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:21.048891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:21.048917 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:47:21.049487 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.049499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.051658 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.051663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.054924 (+ 3261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.054937 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.056995 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.057002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.057300 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.057339 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 20:47:21.057477 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 20:47:21.058799 (+ 1322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27132,"mutex_wait_us":1401,"thread_start_us":26955,"threads_started":1}
I20260926 20:47:21.059650 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.018919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44745 (local address 127.0.0.1:34764)
0926 20:47:21.045353 (+ 26434us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:21.045431 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:21.045723 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:21.048310 (+ 2587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:21.048316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:21.048346 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:47:21.048702 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.048711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.051986 (+ 3275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.051990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.054568 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.054582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.057008 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.057013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.057268 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.057339 (+ 71us) client_negotiation.cc:769] Sending connection context
0926 20:47:21.057477 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 20:47:21.059403 (+ 1926us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26047,"mutex_wait_us":1842,"thread_start_us":28983,"threads_started":1}
I20260926 20:47:21.060258 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.020321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.5.78.3:37661)
0926 20:47:21.044236 (+ 23915us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.044250 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.047880 (+ 3630us) server_negotiation.cc:407] Connection header received
0926 20:47:21.047994 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.048004 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.048243 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.048425 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:47:21.049555 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.051536 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.054586 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.056337 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.056535 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.056767 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.058400 (+ 1633us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.058607 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23625,"thread_start_us":23970,"threads_started":1}
I20260926 20:47:21.063177 15919 tablet_service.cc:1467] Processing CreateTablet for tablet 29acf360681a4d589724d45d8317587c (DEFAULT_TABLE table=test-table [id=14e45b3ccffd449abbcf11ee14236b40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:47:21.063038 16053 tablet_service.cc:1467] Processing CreateTablet for tablet 29acf360681a4d589724d45d8317587c (DEFAULT_TABLE table=test-table [id=14e45b3ccffd449abbcf11ee14236b40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:47:21.064666 15919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29acf360681a4d589724d45d8317587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:47:21.064795 16053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29acf360681a4d589724d45d8317587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:47:21.071805 16187 tablet_service.cc:1467] Processing CreateTablet for tablet 29acf360681a4d589724d45d8317587c (DEFAULT_TABLE table=test-table [id=14e45b3ccffd449abbcf11ee14236b40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:47:21.073562 16187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29acf360681a4d589724d45d8317587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:47:21.079246 16271 tablet_bootstrap.cc:492] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Bootstrap starting.
I20260926 20:47:21.082890 16271 tablet_bootstrap.cc:654] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Neither blocks nor log segments found. Creating new log.
I20260926 20:47:21.084362 16271 log.cc:824] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:47:21.091739 16274 tablet_bootstrap.cc:492] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Bootstrap starting.
I20260926 20:47:21.095201 16273 tablet_bootstrap.cc:492] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Bootstrap starting.
I20260926 20:47:21.097141 16271 tablet_bootstrap.cc:492] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: No bootstrap required, opened a new log
I20260926 20:47:21.097432 16271 ts_tablet_manager.cc:1397] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.004s
I20260926 20:47:21.098783 16273 tablet_bootstrap.cc:654] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Neither blocks nor log segments found. Creating new log.
I20260926 20:47:21.100072 16273 log.cc:824] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Log is configured to *not* fsync() on all Append() calls
I20260926 20:47:21.103471 16273 tablet_bootstrap.cc:492] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: No bootstrap required, opened a new log
I20260926 20:47:21.103750 16273 ts_tablet_manager.cc:1397] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:47:21.107448 16274 tablet_bootstrap.cc:654] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Neither blocks nor log segments found. Creating new log.
I20260926 20:47:21.107926 16271 raft_consensus.cc:348] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.108421 16271 raft_consensus.cc:374] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:47:21.108520 16271 raft_consensus.cc:729] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee2a3888c1e492b822eb7f3a200a0d9, State: Initialized, Role: FOLLOWER
I20260926 20:47:21.109210 16271 consensus_queue.cc:260] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.109411 16274 log.cc:824] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Log is configured to *not* fsync() on all Append() calls
I20260926 20:47:21.115808 16273 raft_consensus.cc:348] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.116315 16273 raft_consensus.cc:374] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:47:21.116438 16273 raft_consensus.cc:729] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfb58f47da9485095c430ca76f83c99, State: Initialized, Role: FOLLOWER
I20260926 20:47:21.116568 16271 ts_tablet_manager.cc:1428] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Time spent starting tablet: real 0.019s user 0.020s sys 0.001s
I20260926 20:47:21.117168 16273 consensus_queue.cc:260] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.118248 16118 heartbeater.cc:503] Master 127.5.78.62:45671 was elected leader, sending a full tablet report...
I20260926 20:47:21.118814 16273 ts_tablet_manager.cc:1428] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260926 20:47:21.119925 16274 tablet_bootstrap.cc:492] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: No bootstrap required, opened a new log
I20260926 20:47:21.120234 16274 ts_tablet_manager.cc:1397] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.012s
W20260926 20:47:21.123538 15985 tablet.cc:2369] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:47:21.128422 16274 raft_consensus.cc:348] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.128854 16274 raft_consensus.cc:374] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:47:21.128981 16274 raft_consensus.cc:729] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930b8a3f1c36420ab0dff42f96faa814, State: Initialized, Role: FOLLOWER
I20260926 20:47:21.129618 16274 consensus_queue.cc:260] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.130875 16252 heartbeater.cc:503] Master 127.5.78.62:45671 was elected leader, sending a full tablet report...
I20260926 20:47:21.131610 16274 ts_tablet_manager.cc:1428] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
W20260926 20:47:21.170697 16119 tablet.cc:2369] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:47:21.189914 16253 tablet.cc:2369] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:47:21.308506 16279 raft_consensus.cc:482] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:47:21.308777 16279 raft_consensus.cc:504] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.310392 16279 leader_election.cc:288] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee2a3888c1e492b822eb7f3a200a0d9 (127.5.78.1:44745), 4dfb58f47da9485095c430ca76f83c99 (127.5.78.2:36455)
I20260926 20:47:21.316936 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.310775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44745 (local address 127.5.78.3:36301)
0926 20:47:21.311145 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:21.311174 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:21.311333 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:21.311764 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:21.311767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:21.311807 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:47:21.311991 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.312006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.313245 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.313250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.314770 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.314782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.316123 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.316129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.316311 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.316347 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:47:21.316467 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 20:47:21.316602 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260926 20:47:21.316967 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.310854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:36301 (local address 127.5.78.1:44745)
0926 20:47:21.311026 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.311034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.311280 (+ 246us) server_negotiation.cc:407] Connection header received
0926 20:47:21.311499 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.311506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.311589 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.311704 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:47:21.312138 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.313098 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.314920 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.315970 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.316070 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.316214 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.316492 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.316693 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:47:21.317800 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.310806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:43435 (local address 127.5.78.2:36455)
0926 20:47:21.311009 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.311015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.311710 (+ 695us) server_negotiation.cc:407] Connection header received
0926 20:47:21.311857 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.311862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.311932 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.312026 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:47:21.312703 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.313540 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.315632 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.316867 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.317005 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.317096 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.317503 (+ 407us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.317633 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:47:21.317931 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.310800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:36455 (local address 127.5.78.3:43435)
0926 20:47:21.311572 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0926 20:47:21.311589 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:47:21.311731 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:47:21.312054 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:47:21.312076 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:47:21.312102 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:47:21.312546 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.312558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.313701 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.313707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.315427 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:47:21.315440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.317223 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.317230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:47:21.317350 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.317372 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 20:47:21.317446 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 20:47:21.317552 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":21,"thread_start_us":403,"threads_started":1}
I20260926 20:47:21.318068 15939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acf360681a4d589724d45d8317587c" candidate_uuid: "930b8a3f1c36420ab0dff42f96faa814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" is_pre_election: true
I20260926 20:47:21.318660 15939 raft_consensus.cc:2454] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 930b8a3f1c36420ab0dff42f96faa814 in term 0.
I20260926 20:47:21.318894 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acf360681a4d589724d45d8317587c" candidate_uuid: "930b8a3f1c36420ab0dff42f96faa814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dfb58f47da9485095c430ca76f83c99" is_pre_election: true
I20260926 20:47:21.319348 16140 leader_election.cc:302] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee2a3888c1e492b822eb7f3a200a0d9, 930b8a3f1c36420ab0dff42f96faa814; no voters:
I20260926 20:47:21.319406 16073 raft_consensus.cc:2454] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 930b8a3f1c36420ab0dff42f96faa814 in term 0.
I20260926 20:47:21.319739 16279 raft_consensus.cc:2781] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:47:21.319895 16279 raft_consensus.cc:482] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:47:21.320029 16279 raft_consensus.cc:3037] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:47:21.322134 16279 raft_consensus.cc:504] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.322845 16279 leader_election.cc:288] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee2a3888c1e492b822eb7f3a200a0d9 (127.5.78.1:44745), 4dfb58f47da9485095c430ca76f83c99 (127.5.78.2:36455)
I20260926 20:47:21.323201 15939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acf360681a4d589724d45d8317587c" candidate_uuid: "930b8a3f1c36420ab0dff42f96faa814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9"
I20260926 20:47:21.323323 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29acf360681a4d589724d45d8317587c" candidate_uuid: "930b8a3f1c36420ab0dff42f96faa814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dfb58f47da9485095c430ca76f83c99"
I20260926 20:47:21.323397 15939 raft_consensus.cc:3037] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:47:21.323552 16073 raft_consensus.cc:3037] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:47:21.325537 15939 raft_consensus.cc:2454] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 930b8a3f1c36420ab0dff42f96faa814 in term 1.
I20260926 20:47:21.325760 16073 raft_consensus.cc:2454] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 930b8a3f1c36420ab0dff42f96faa814 in term 1.
I20260926 20:47:21.325958 16140 leader_election.cc:302] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee2a3888c1e492b822eb7f3a200a0d9, 930b8a3f1c36420ab0dff42f96faa814; no voters:
I20260926 20:47:21.326251 16279 raft_consensus.cc:2781] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:47:21.326787 16279 raft_consensus.cc:686] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 1 LEADER]: Becoming Leader. State: Replica: 930b8a3f1c36420ab0dff42f96faa814, State: Running, Role: LEADER
I20260926 20:47:21.327281 16279 consensus_queue.cc:237] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } }
I20260926 20:47:21.331983 15794 catalog_manager.cc:5495] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 reported cstate change: term changed from 0 to 1, leader changed from <none> to 930b8a3f1c36420ab0dff42f96faa814 (127.5.78.3). New cstate: current_term: 1 leader_uuid: "930b8a3f1c36420ab0dff42f96faa814" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "930b8a3f1c36420ab0dff42f96faa814" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 37661 } health_report { overall_health: HEALTHY } } }
I20260926 20:47:21.354801 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.349402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.5.78.3:37661)
0926 20:47:21.349582 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.349587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.349605 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:47:21.349691 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.349694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.349771 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.349907 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:47:21.350269 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.351120 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.352504 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.352919 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.353023 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.353539 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:47:21.354076 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:47:21.354186 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.354358 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.354494 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:47:21.363260 15939 raft_consensus.cc:1261] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 1 FOLLOWER]: Refusing update from remote peer 930b8a3f1c36420ab0dff42f96faa814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:47:21.363261 16073 raft_consensus.cc:1261] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 1 FOLLOWER]: Refusing update from remote peer 930b8a3f1c36420ab0dff42f96faa814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:47:21.364219 16279 consensus_queue.cc:1035] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee2a3888c1e492b822eb7f3a200a0d9" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:47:21.364555 16281 consensus_queue.cc:1035] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dfb58f47da9485095c430ca76f83c99" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:47:21.381917 16285 mvcc.cc:205] Tried to move back new op lower bound from 7333706307012726784 to 7333706306879827968. Current Snapshot: MvccSnapshot[applied={T|T < 7333706307012726784}]
I20260926 20:47:21.392258 15795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:47:21.392554 15795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:47:21.396138 15795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce8dcd7905ef4ff4bfd0939eed437721: Sending DeleteTablet for 3 replicas of tablet 29acf360681a4d589724d45d8317587c
I20260926 20:47:21.397310 15919 tablet_service.cc:1514] Processing DeleteTablet for tablet 29acf360681a4d589724d45d8317587c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:47:21 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:34764
I20260926 20:47:21.397349 16053 tablet_service.cc:1514] Processing DeleteTablet for tablet 29acf360681a4d589724d45d8317587c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:47:21 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:60850
I20260926 20:47:21.397672 16187 tablet_service.cc:1514] Processing DeleteTablet for tablet 29acf360681a4d589724d45d8317587c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:47:21 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:42638
I20260926 20:47:21.400182 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15856
I20260926 20:47:21.431658 16295 tablet_replica.cc:330] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: stopping tablet replica
I20260926 20:47:21.432125 16295 raft_consensus.cc:2229] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:47:21.432654 16295 raft_consensus.cc:2258] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:47:21.433506 16296 tablet_replica.cc:330] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: stopping tablet replica
I20260926 20:47:21.433960 16296 raft_consensus.cc:2229] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:47:21.434621 16296 raft_consensus.cc:2258] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:47:21.435489 16294 tablet_replica.cc:330] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: stopping tablet replica
I20260926 20:47:21.435878 16294 raft_consensus.cc:2229] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:47:21.436098 16295 ts_tablet_manager.cc:1905] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:47:21.436429 16294 raft_consensus.cc:2258] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:47:21.438364 16294 ts_tablet_manager.cc:1905] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:47:21.439883 16296 ts_tablet_manager.cc:1905] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:47:21.445230 16294 ts_tablet_manager.cc:1918] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:47:21.445400 16294 log.cc:1196] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/29acf360681a4d589724d45d8317587c
Sep 26 20:47:21 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:47:21.445914 16294 ts_tablet_manager.cc:1939] T 29acf360681a4d589724d45d8317587c P 4dfb58f47da9485095c430ca76f83c99: Deleting consensus metadata
I20260926 20:47:21.448104 15782 catalog_manager.cc:4841] TS 4dfb58f47da9485095c430ca76f83c99 (127.5.78.2:36455): tablet 29acf360681a4d589724d45d8317587c (table test-table [id=14e45b3ccffd449abbcf11ee14236b40]) successfully deleted
I20260926 20:47:21.452373 16295 ts_tablet_manager.cc:1918] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:47:21.452776 16295 log.cc:1196] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/29acf360681a4d589724d45d8317587c
I20260926 20:47:21.453382 16295 ts_tablet_manager.cc:1939] T 29acf360681a4d589724d45d8317587c P 4ee2a3888c1e492b822eb7f3a200a0d9: Deleting consensus metadata
I20260926 20:47:21.453505 16296 ts_tablet_manager.cc:1918] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:47:21.453667 16296 log.cc:1196] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/29acf360681a4d589724d45d8317587c
I20260926 20:47:21.454229 16296 ts_tablet_manager.cc:1939] T 29acf360681a4d589724d45d8317587c P 930b8a3f1c36420ab0dff42f96faa814: Deleting consensus metadata
I20260926 20:47:21.455224 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.401062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.5.78.1:44745)
0926 20:47:21.401238 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.401245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.426319 (+ 25074us) server_negotiation.cc:407] Connection header received
0926 20:47:21.426587 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.426594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.426706 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.426844 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:21.428662 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.429925 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.433333 (+ 3408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.434064 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.434178 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.447002 (+ 12824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:21.447019 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:21.447040 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:21.447085 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:21.451436 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:21.452443 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:21.452454 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:21.452457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:21.452633 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:21.453147 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:21.453159 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:21.453163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:21.453619 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:21.453931 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.454553 (+ 622us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.454806 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:47:21.455921 15779 catalog_manager.cc:4841] TS 4ee2a3888c1e492b822eb7f3a200a0d9 (127.5.78.1:44745): tablet 29acf360681a4d589724d45d8317587c (table test-table [id=14e45b3ccffd449abbcf11ee14236b40]) successfully deleted
I20260926 20:47:21.456710 15781 catalog_manager.cc:4841] TS 930b8a3f1c36420ab0dff42f96faa814 (127.5.78.3:37661): tablet 29acf360681a4d589724d45d8317587c (table test-table [id=14e45b3ccffd449abbcf11ee14236b40]) successfully deleted
I20260926 20:47:21.456881 15979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:21.567751 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15856
I20260926 20:47:21.587798 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15990
Sep 26 20:47:21 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:47:21.609884 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.588751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.5.78.2:36455)
0926 20:47:21.588956 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.588963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.588986 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:47:21.589079 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.589085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.589176 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.589292 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:21.590735 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.591908 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.593158 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.593786 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.593916 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.602251 (+ 8335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:21.602274 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:21.602297 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:21.602353 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:21.606346 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:21.607424 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:21.607435 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:21.607439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:21.607622 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:21.608109 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:21.608115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:21.608118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:21.608618 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:21.608928 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.609256 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.609497 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:47:21.610657 16113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:21.703893 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15990
I20260926 20:47:21.727526 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 16124
Sep 26 20:47:21 dist-test-slave-z7j8 krb5kdc[15750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:47:21.746013 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:47:21.728401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.5.78.3:37661)
0926 20:47:21.728598 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:47:21.728604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:47:21.728627 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:47:21.728771 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:47:21.728789 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:47:21.728888 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:47:21.729016 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:47:21.730270 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.731567 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.732841 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:47:21.733531 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:47:21.733692 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:47:21.738557 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:47:21.738579 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:47:21.738601 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:47:21.738669 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:47:21.742973 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:21.743768 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:21.743785 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:21.743789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:21.743999 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:47:21.744364 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:47:21.744371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:47:21.744376 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:47:21.744854 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:47:21.745187 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 20:47:21.745368 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 20:47:21.745594 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:47:21.746870 16247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:21.866606 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 16124
I20260926 20:47:21.888841 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 15763
I20260926 20:47:21.890156 15825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:47:21.998638 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 15763
2026-09-26T20:47:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4584 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:47:22 dist-test-slave-z7j8 krb5kdc[16304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:47:22 dist-test-slave-z7j8 krb5kdc[16304](info): set up 2 sockets
Sep 26 20:47:22 dist-test-slave-z7j8 krb5kdc[16304](info): commencing operation
krb5kdc: starting...
W20260926 20:47:23.099205 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:47:23 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:47:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/rangeradmin_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/rangeradmin_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/rangerlookup_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/rangerlookup_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
I20260926 20:47:23.442267 5432 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskb6fvOW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/postgres -l logfile start
2026-09-26 20:47:26.138 UTC [16334] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 20:47:26.138 UTC [16334] LOG: listening on IPv4 address "127.5.78.20", port 48053
2026-09-26 20:47:26.141 UTC [16334] LOG: listening on Unix socket "/tmp/.s.PGSQL.48053"
2026-09-26 20:47:26.166 UTC [16337] LOG: database system was shut down at 2026-09-26 20:47:24 UTC
2026-09-26 20:47:26.170 UTC [16334] LOG: database system is ready to accept connections
I20260926 20:47:27.143486 5432 mini_postgres.cc:96] Postgres bound to 48053
2026-09-26 20:47:27.161 UTC [16345] FATAL: database "slave" does not exist
127.5.78.20:48053 - accepting connections
I20260926 20:47:27.162395 5432 mini_ranger.cc:161] Starting Ranger...
I20260926 20:47:27.190456 5432 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 20:47:27.706313 5432 mini_ranger.cc:87] Created ranger Postgres database
I20260926 20:47:27.706470 5432 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-admin
2026-09-26 20:47:28,010 [I] DB FLAVOR :POSTGRES
2026-09-26 20:47:28,010 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:47:28,010 [I] Checking connection..
2026-09-26 20:47:28,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 20:47:28,276 [I] Checking connection passed.
2026-09-26 20:47:28,276 [I] --------- Verifying version history table ---------
2026-09-26 20:47:28,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:47:28,520 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:47:28,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:47:28,802 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:47:28,803 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 20:47:28,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 20:47:29.050 UTC [16409] WARNING: there is no transaction in progress
2026-09-26 20:47:29,056 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 20:47:29,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:47:29,299 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 20:47:29,299 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:47:29,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 20:47:29,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 20:47:29,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z7j8.c.gcp-upstream.internal','N') ;"
2026-09-26 20:47:30,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:47:30,295 [I] Table x_portal_user does not exist in database ranger
2026-09-26 20:47:30,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:47:30,539 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 20:47:30,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:47:30,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:47:31,036 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 20:47:31,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 20:47:32.105 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.123 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.141 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.156 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.175 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.196 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.255 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.262 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.277 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.289 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.301 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.317 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.328 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.335 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.343 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:32.351 UTC [16517] WARNING: there is no transaction in progress
2026-09-26 20:47:33,066 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 20:47:33,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 20:47:33,307 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 20:47:33,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:47:33,580 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 20:47:33,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:47:33,845 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 20:47:33,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:47:34,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:47:34,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z7j8.c.gcp-upstream.internal';"
2026-09-26 20:47:34,593 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 20:47:34,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 20:47:34,841 [I] DB_PATCHES have already been applied
I20260926 20:47:34.847654 5432 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-admin:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskb6fvOW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 20:47:34.847751 5432 mini_ranger.cc:193] Using host: 127.5.78.20
I20260926 20:47:34.863214 5432 mini_ranger.cc:227] Ranger admin URL: http://127.5.78.20:48983
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 8:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 20:47:35.998780 5432 mini_ranger.cc:160] Time spent starting Ranger: real 8.836s user 0.006s sys 0.035s
Sep 26, 2026 8:47:55 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 8:47:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 8:47:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 8:47:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 8:47:56 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 20:47:57.437520 5432 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/rangerkms_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/rangerkms_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.78.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.78.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 20:47:57 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455677, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 20:47:57.604589 5432 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 20:47:57.625406 5432 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 20:47:58.549213 5432 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 20:47:58.549356 5432 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms
I20260926 20:47:58.549518 5432 mini_ranger_kms.cc:227] Using postgres at 127.5.78.20:48053
2026-09-26 20:47:58,957 [I] DB FLAVOR :POSTGRES
2026-09-26 20:47:58,957 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:47:58,957 [I] Checking connection
2026-09-26 20:47:58,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 20:47:59,214 [I] connection success
2026-09-26 20:47:59,214 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 20:47:59,215 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 20:47:59,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 20:47:59,455 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 20:47:59,456 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:47:59,456 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 20:47:59,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:48053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 20:47:59,748 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 20:48:00.262805 5432 mini_ranger_kms.cc:310] Created kms service
I20260926 20:48:00.479604 5432 mini_ranger_kms.cc:326] Created kudu user
I20260926 20:48:00.529120 5432 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 20:48:00 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455677, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:00.982582 5432 mini_ranger_kms.cc:384] Added ranger policy
I20260926 20:48:00.982900 5432 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms:/tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 20:48:00.982954 5432 mini_ranger_kms.cc:240] Using host: 127.5.78.20
I20260926 20:48:01.004451 5432 mini_ranger_kms.cc:276] Ranger KMS PID: 16697
I20260926 20:48:01.004650 5432 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.78.20:54367
Sep 26, 2026 8:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 8:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 8:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 8:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 20:48:03.369285 5432 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.765s user 0.123s sys 0.308s
I20260926 20:48:03.369618 5432 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 20:48:03.369839 5432 mini_ranger_kms.cc:401] 127.5.78.20:54367/kms/v1/keys
Sep 26 20:48:03 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455683, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.78.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 20:48:04 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455683, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.78.20@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
Sep 26 20:48:04 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455683, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.78.20@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
Sep 26 20:48:06 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:48:06.923008 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46737
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:46737
--ranger_config_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:54367/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:48:07.281248 16748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:48:07.281581 16748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:48:07.281746 16748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:48:07.289561 16748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:48:07.289701 16748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:48:07.289793 16748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:48:07.289882 16748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:48:07.289978 16748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:48:07.301090 16748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.78.20:54367/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:46737
--ranger_config_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:48:07.302656 16748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:48:07.304174 16748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:48:07.319075 16756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:07.319115 16754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:07.319406 16753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:48:07.320186 16748 server_base.cc:1034] running on GCE node
I20260926 20:48:07.321063 16748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:48:07.322818 16748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:48:07.324098 16748 hybrid_clock.cc:648] HybridClock initialized: now 1790455687324043 us; error 52 us; skew 500 ppm
I20260926 20:48:07.324592 16748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:48:07 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:48:07.332342 16748 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:48:07.334470 16748 webserver.cc:460] Webserver started at http://127.5.78.62:34487/ using document root <none> and password file <none>
I20260926 20:48:07.335250 16748 fs_manager.cc:359] Metadata directory not provided
I20260926 20:48:07.335402 16748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:48:07.335758 16748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:48:07 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:07.391810 16748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "c2e4f9de7d034527a864c4c69281dad8"
format_stamp: "Formatted at 2026-09-26 20:48:07 on dist-test-slave-z7j8"
server_key: "99457aa8a75edd36f3e38e9dcaad2872"
server_key_iv: "db1c91db6962ebb8ad118fd83a3d0d3d"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:07.392913 16748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "c2e4f9de7d034527a864c4c69281dad8"
format_stamp: "Formatted at 2026-09-26 20:48:07 on dist-test-slave-z7j8"
server_key: "99457aa8a75edd36f3e38e9dcaad2872"
server_key_iv: "db1c91db6962ebb8ad118fd83a3d0d3d"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:07.399888 16748 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:48:07.419241 16763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:07.421036 16748 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:48:07.421280 16748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c2e4f9de7d034527a864c4c69281dad8"
format_stamp: "Formatted at 2026-09-26 20:48:07 on dist-test-slave-z7j8"
server_key: "99457aa8a75edd36f3e38e9dcaad2872"
server_key_iv: "db1c91db6962ebb8ad118fd83a3d0d3d"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:07.421540 16748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:48:07.440696 16748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:48:07.445556 16748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:48:07.446008 16748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:48:07.468869 16748 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:46737
I20260926 20:48:07.468869 16814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:46737 every 8 connection(s)
I20260926 20:48:07.470614 16748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:48:07.475484 16815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:48:07.477118 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 16748
I20260926 20:48:07.477360 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:48:07.478186 5432 external_mini_cluster.cc:1428] Setting key b36f50828d74f71cd9c9a4b7e0870258
I20260926 20:48:07.493929 16815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8: Bootstrap starting.
I20260926 20:48:07.498009 16815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8: Neither blocks nor log segments found. Creating new log.
I20260926 20:48:07.499550 16815 log.cc:824] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8: Log is configured to *not* fsync() on all Append() calls
I20260926 20:48:07.503146 16815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8: No bootstrap required, opened a new log
Sep 26 20:48:07 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:48:07.514570 16815 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e4f9de7d034527a864c4c69281dad8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46737 } }
I20260926 20:48:07.515127 16815 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:48:07.515300 16815 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e4f9de7d034527a864c4c69281dad8, State: Initialized, Role: FOLLOWER
I20260926 20:48:07.516042 16815 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e4f9de7d034527a864c4c69281dad8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46737 } }
I20260926 20:48:07.516345 16815 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:48:07.516517 16815 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:48:07.516743 16815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:48:07.520051 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:07.480591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.5.78.62:46737)
0926 20:48:07.492703 (+ 12112us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:07.492722 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:07.492780 (+ 58us) server_negotiation.cc:407] Connection header received
0926 20:48:07.495320 (+ 2540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:07.495355 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:07.495966 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:07.496508 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:07.498290 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:07.500403 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:07.502139 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:07.503219 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:07.503398 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:07.508536 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:07.508553 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:07.508578 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:07.508640 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:07.513490 (+ 4850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:07.514856 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:07.514867 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:07.514876 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:07.515118 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:07.515837 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:07.515845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:07.515849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:07.516786 (+ 937us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:07.517156 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:07.517976 (+ 820us) server_negotiation.cc:299] Negotiation successful
0926 20:48:07.518526 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11793,"thread_start_us":11744,"threads_started":1}
I20260926 20:48:07.519991 16815 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e4f9de7d034527a864c4c69281dad8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46737 } }
I20260926 20:48:07.520682 16815 leader_election.cc:302] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2e4f9de7d034527a864c4c69281dad8; no voters:
I20260926 20:48:07.521461 16815 leader_election.cc:288] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:48:07.521621 16820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:48:07.528587 16815 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:48:07.538873 16820 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [term 1 LEADER]: Becoming Leader. State: Replica: c2e4f9de7d034527a864c4c69281dad8, State: Running, Role: LEADER
I20260926 20:48:07.539925 16820 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e4f9de7d034527a864c4c69281dad8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46737 } }
I20260926 20:48:07.545818 16815 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 20:48:07.545900 16821 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2e4f9de7d034527a864c4c69281dad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e4f9de7d034527a864c4c69281dad8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46737 } } }
I20260926 20:48:07.545989 16822 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2e4f9de7d034527a864c4c69281dad8. Latest consensus state: current_term: 1 leader_uuid: "c2e4f9de7d034527a864c4c69281dad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e4f9de7d034527a864c4c69281dad8" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46737 } } }
I20260926 20:48:07.546319 16821 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [sys.catalog]: This master's current role is: LEADER
I20260926 20:48:07.546351 16822 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8 [sys.catalog]: This master's current role is: LEADER
I20260926 20:48:07.547262 16834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:48:07.567067 16834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:48:07.595785 16834 catalog_manager.cc:1347] Generated new cluster ID: 22ca4d03860d47da8df3b503fe7c75f0
I20260926 20:48:07.595937 16834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:48:07.613945 16834 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:48:07.616380 16834 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:48:07.627076 16834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8: Generated new TSK 0
I20260926 20:48:07.628465 16834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:48:08.443585 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:54367/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46737
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:48:08 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 20:48:08.914281 16860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:48:08.914742 16860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:48:08.914924 16860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:48:08.924145 16860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:48:08.924381 16860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:48:08.924619 16860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:48:08.937486 16860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.78.20:54367/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:48:08.939575 16860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:48:08.941546 16860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:48:08.965507 16871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:08.965507 16870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:08.975675 16873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:48:08.975956 16860 server_base.cc:1034] running on GCE node
I20260926 20:48:08.976778 16860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:48:08.978003 16860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:48:08.979379 16860 hybrid_clock.cc:648] HybridClock initialized: now 1790455688979301 us; error 73 us; skew 500 ppm
I20260926 20:48:08.979826 16860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:48:08 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:48:08.983767 16860 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:48:08.990715 16860 webserver.cc:460] Webserver started at http://127.5.78.1:43641/ using document root <none> and password file <none>
I20260926 20:48:08.991614 16860 fs_manager.cc:359] Metadata directory not provided
I20260926 20:48:08.991799 16860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:48:08.992180 16860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:48:08 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:09.015255 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "5b041db2c93e4c9bb44fd9653fe070b4"
format_stamp: "Formatted at 2026-09-26 20:48:09 on dist-test-slave-z7j8"
server_key: "7bb0bdd7a682edff21b2432c91cfb82d"
server_key_iv: "99595624513a84de456ef890f2e8cac9"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:09.016822 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "5b041db2c93e4c9bb44fd9653fe070b4"
format_stamp: "Formatted at 2026-09-26 20:48:09 on dist-test-slave-z7j8"
server_key: "7bb0bdd7a682edff21b2432c91cfb82d"
server_key_iv: "99595624513a84de456ef890f2e8cac9"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:09.039659 16860 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.017s sys 0.000s
I20260926 20:48:09.063933 16880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:09.066175 16860 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.003s
I20260926 20:48:09.066556 16860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "5b041db2c93e4c9bb44fd9653fe070b4"
format_stamp: "Formatted at 2026-09-26 20:48:09 on dist-test-slave-z7j8"
server_key: "7bb0bdd7a682edff21b2432c91cfb82d"
server_key_iv: "99595624513a84de456ef890f2e8cac9"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:09.066934 16860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:48:09.092782 16860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:48:09.099694 16860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:48:09.100594 16860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:48:09.103322 16860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:48:09.107172 16860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:48:09.107357 16860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 20:48:09.107504 16860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:48:09.107574 16860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:48:09 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:09.440431 16860 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:39667
I20260926 20:48:09.442526 16860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:48:09.440447 16992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:39667 every 8 connection(s)
I20260926 20:48:09.446784 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 16860
I20260926 20:48:09.446995 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:48:09.447751 5432 external_mini_cluster.cc:1428] Setting key 519a97fd8ca8c7d50b986906bbe59207
Sep 26 20:48:09 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:48:09.495683 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:09.453512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:33401 (local address 127.5.78.62:46737)
0926 20:48:09.467399 (+ 13887us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:09.467408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:09.470481 (+ 3073us) server_negotiation.cc:407] Connection header received
0926 20:48:09.474461 (+ 3980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:09.474467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:09.474564 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:09.474650 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:09.477313 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:09.478936 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:09.481816 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:09.482339 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:09.482519 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:09.487724 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:09.487735 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:09.487743 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:09.487795 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:09.491689 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:09.492794 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:09.492802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:09.492805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:09.492951 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:09.493576 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:09.493582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:09.493585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:09.493952 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:09.494088 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:09.495022 (+ 934us) server_negotiation.cc:299] Negotiation successful
0926 20:48:09.495290 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13697,"thread_start_us":13548,"threads_started":1}
I20260926 20:48:09.496806 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:09.452805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46737 (local address 127.5.78.1:33401)
0926 20:48:09.469581 (+ 16776us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:09.469656 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:09.473918 (+ 4262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:09.475351 (+ 1433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:09.475367 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:09.476066 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:48:09.477036 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:09.477069 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:09.479233 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:09.479238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:09.481573 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:09.481595 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:09.482928 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:09.482938 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:09.483302 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:09.484221 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:48:09.484264 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:48:09.487393 (+ 3129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:48:09.491909 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:48:09.491918 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:48:09.491938 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:48:09.492631 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:48:09.493104 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:48:09.493109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:48:09.493112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:48:09.493430 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:48:09.494203 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:48:09.494209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:48:09.494648 (+ 439us) client_negotiation.cc:769] Sending connection context
0926 20:48:09.494950 (+ 302us) client_negotiation.cc:240] Negotiation successful
0926 20:48:09.495537 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14905,"thread_start_us":14667,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:48:09.498631 16993 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46737
I20260926 20:48:09.498976 16993 heartbeater.cc:464] Registering TS with master...
I20260926 20:48:09.499771 16993 heartbeater.cc:511] Master 127.5.78.62:46737 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:48:09.502107 16780 ts_manager.cc:194] Registered new tserver with Master: 5b041db2c93e4c9bb44fd9653fe070b4 (127.5.78.1:39667)
I20260926 20:48:09.503686 16780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:33401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:48:09.538971 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:54367/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46737
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:48:09 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
W20260926 20:48:10.044689 16999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:48:10.045123 16999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:48:10.045275 16999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:48:10.057327 16999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:48:10.057513 16999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:48:10.057694 16999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:48:10.074023 16999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.78.20:54367/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:48:10.075893 16999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:48:10.077726 16999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:48:10.095021 17014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:10.095130 17013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:10.095738 17016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:48:10.096756 16999 server_base.cc:1034] running on GCE node
I20260926 20:48:10.097591 16999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:48:10.098991 16999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:48:10.100276 16999 hybrid_clock.cc:648] HybridClock initialized: now 1790455690100220 us; error 42 us; skew 500 ppm
I20260926 20:48:10.100690 16999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:48:10 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455690, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:48:10.105263 16999 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:48:10.107301 16999 webserver.cc:460] Webserver started at http://127.5.78.2:38713/ using document root <none> and password file <none>
I20260926 20:48:10.108265 16999 fs_manager.cc:359] Metadata directory not provided
I20260926 20:48:10.108412 16999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:48:10.108805 16999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:48:10 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455690, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:10.128541 16999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "9e90b1496152492b88fff5fe00574a86"
format_stamp: "Formatted at 2026-09-26 20:48:10 on dist-test-slave-z7j8"
server_key: "ea35e37653eb004aa2a8f6e27da7d9a0"
server_key_iv: "93c8dda31ca03bcde7346eb4fba51372"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:10.130054 16999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "9e90b1496152492b88fff5fe00574a86"
format_stamp: "Formatted at 2026-09-26 20:48:10 on dist-test-slave-z7j8"
server_key: "ea35e37653eb004aa2a8f6e27da7d9a0"
server_key_iv: "93c8dda31ca03bcde7346eb4fba51372"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:10.138705 16999 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.003s
I20260926 20:48:10.157927 17023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:10.160110 16999 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260926 20:48:10.160375 16999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "9e90b1496152492b88fff5fe00574a86"
format_stamp: "Formatted at 2026-09-26 20:48:10 on dist-test-slave-z7j8"
server_key: "ea35e37653eb004aa2a8f6e27da7d9a0"
server_key_iv: "93c8dda31ca03bcde7346eb4fba51372"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:10.160723 16999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:48:10.188876 16999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:48:10.195012 16999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:48:10.195873 16999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:48:10.197220 16999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:48:10.199261 16999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:48:10.199432 16999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 20:48:10.199599 16999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:48:10.199712 16999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:10.244841 16999 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:38411
I20260926 20:48:10.244886 17135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:38411 every 8 connection(s)
I20260926 20:48:10.246670 16999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:48:10.248431 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 16999
I20260926 20:48:10.248616 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:48:10.249151 5432 external_mini_cluster.cc:1428] Setting key c01fc95c79c12a608882dcc8578df38a
Sep 26 20:48:10 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455690, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:48:10.283380 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.251645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:56397 (local address 127.5.78.62:46737)
0926 20:48:10.258947 (+ 7302us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:10.258953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:10.262360 (+ 3407us) server_negotiation.cc:407] Connection header received
0926 20:48:10.262476 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:10.262485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:10.262593 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:10.262734 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:10.265476 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.266891 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.269402 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.270229 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.270490 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.275321 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:10.275331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:10.275340 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:10.275395 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:10.279456 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:10.280498 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:10.280505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:10.280508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:10.280676 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:10.281233 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:10.281238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:10.281241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:10.281674 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:10.281817 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:10.282755 (+ 938us) server_negotiation.cc:299] Negotiation successful
0926 20:48:10.282999 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7143,"thread_start_us":7023,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:48:10.284653 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.252202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46737 (local address 127.5.78.2:56397)
0926 20:48:10.259622 (+ 7420us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:10.259696 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:10.262076 (+ 2380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:10.263034 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:10.263049 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:10.263804 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:48:10.265177 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.265210 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.267117 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.267123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.269181 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.269202 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.270613 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.270620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.270965 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.271869 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:48:10.271901 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:48:10.274983 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:48:10.279760 (+ 4777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:48:10.279775 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:48:10.279801 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:48:10.280329 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:48:10.280904 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:48:10.280910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:48:10.280913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:48:10.281107 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:48:10.281857 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:48:10.281864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:48:10.282213 (+ 349us) client_negotiation.cc:769] Sending connection context
0926 20:48:10.282614 (+ 401us) client_negotiation.cc:240] Negotiation successful
0926 20:48:10.283049 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7057,"thread_start_us":6803,"threads_started":1}
I20260926 20:48:10.291126 17136 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46737
I20260926 20:48:10.291517 17136 heartbeater.cc:464] Registering TS with master...
I20260926 20:48:10.292424 17136 heartbeater.cc:511] Master 127.5.78.62:46737 requested a full tablet report, sending...
I20260926 20:48:10.294395 16780 ts_manager.cc:194] Registered new tserver with Master: 9e90b1496152492b88fff5fe00574a86 (127.5.78.2:38411)
I20260926 20:48:10.295729 16780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:56397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:48:10.314990 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:54367/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46737
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:48:10.507083 16993 heartbeater.cc:503] Master 127.5.78.62:46737 was elected leader, sending a full tablet report...
W20260926 20:48:10.703032 17142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:48:10.703368 17142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:48:10.703552 17142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:48:10.712355 17142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:48:10.712507 17142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:48:10.712702 17142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:48:10.723017 17142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.78.20:54367/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:48:10.724378 17142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:48:10.726145 17142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:48:10.740748 17147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:10.740688 17150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:10.740813 17148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:48:10.741947 17142 server_base.cc:1034] running on GCE node
I20260926 20:48:10.742693 17142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:48:10.743734 17142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:48:10.745009 17142 hybrid_clock.cc:648] HybridClock initialized: now 1790455690744945 us; error 78 us; skew 500 ppm
I20260926 20:48:10.745399 17142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:48:10 dist-test-slave-z7j8 krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455690, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:48:10.749338 17142 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:48:10.751116 17142 webserver.cc:460] Webserver started at http://127.5.78.3:35759/ using document root <none> and password file <none>
I20260926 20:48:10.751793 17142 fs_manager.cc:359] Metadata directory not provided
I20260926 20:48:10.751897 17142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:48:10.752161 17142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:48:10 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455690, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:10.767774 17142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae"
format_stamp: "Formatted at 2026-09-26 20:48:10 on dist-test-slave-z7j8"
server_key: "b37c7f2fb438d3273a1baeea2f5c6b29"
server_key_iv: "6a1340893f8000cfb3feebe5be327bf9"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:10.768770 17142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae"
format_stamp: "Formatted at 2026-09-26 20:48:10 on dist-test-slave-z7j8"
server_key: "b37c7f2fb438d3273a1baeea2f5c6b29"
server_key_iv: "6a1340893f8000cfb3feebe5be327bf9"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:10.775370 17142 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 20:48:10.795105 17157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:10.796957 17142 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.001s
I20260926 20:48:10.797190 17142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae"
format_stamp: "Formatted at 2026-09-26 20:48:10 on dist-test-slave-z7j8"
server_key: "b37c7f2fb438d3273a1baeea2f5c6b29"
server_key_iv: "6a1340893f8000cfb3feebe5be327bf9"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:10.797423 17142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:48:10.821966 17142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:48:10.826651 17142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:48:10.827083 17142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:48:10.828190 17142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:48:10.829882 17142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:48:10.829967 17142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:10.830060 17142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:48:10.830106 17142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:10.869626 17142 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:36831
I20260926 20:48:10.869665 17269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:36831 every 8 connection(s)
I20260926 20:48:10.870985 17142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:48:10.873188 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 17142
I20260926 20:48:10.873383 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:48:10.873929 5432 external_mini_cluster.cc:1428] Setting key 995655059e12f90d103184c005764103
Sep 26 20:48:10 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455690, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:48:10.896600 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.874625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:33989 (local address 127.5.78.62:46737)
0926 20:48:10.875400 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:10.875406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:10.876493 (+ 1087us) server_negotiation.cc:407] Connection header received
0926 20:48:10.878813 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:10.878817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:10.878903 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:10.879007 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:10.880715 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.881469 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.883423 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.883885 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.884026 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.888701 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:10.888708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:10.888713 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:10.888761 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:10.892719 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:10.893709 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:10.893714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:10.893716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:10.893992 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:10.894791 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:10.894795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:10.894798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:10.895123 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:10.895271 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:10.896088 (+ 817us) server_negotiation.cc:299] Negotiation successful
0926 20:48:10.896319 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":510,"threads_started":1}
I20260926 20:48:10.897712 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.875246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46737 (local address 127.5.78.3:33989)
0926 20:48:10.876295 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:10.876350 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:10.878368 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:10.879298 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:10.879307 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:10.879786 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:48:10.880470 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.880492 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.881694 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.881698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.883222 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.883238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.884154 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.884160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.884406 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.885262 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:48:10.885293 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:48:10.888219 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:48:10.892978 (+ 4759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:48:10.892987 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:48:10.893004 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:48:10.893518 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:48:10.894202 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:48:10.894206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:48:10.894208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:48:10.894586 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:48:10.895296 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:48:10.895304 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:48:10.895672 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 20:48:10.895954 (+ 282us) client_negotiation.cc:240] Negotiation successful
0926 20:48:10.896382 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":449,"threads_started":1}
I20260926 20:48:10.899675 17270 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46737
I20260926 20:48:10.900079 17270 heartbeater.cc:464] Registering TS with master...
I20260926 20:48:10.900924 17270 heartbeater.cc:511] Master 127.5.78.62:46737 requested a full tablet report, sending...
I20260926 20:48:10.902822 16780 ts_manager.cc:194] Registered new tserver with Master: 3576c5f5d4104af9bdfb1cd9c0f887ae (127.5.78.3:36831)
I20260926 20:48:10.903867 16780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:33989
I20260926 20:48:10.904327 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:48:10.923662 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.909297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.5.78.62:46737)
0926 20:48:10.909522 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:10.909527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:10.909892 (+ 365us) server_negotiation.cc:407] Connection header received
0926 20:48:10.910057 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:10.910061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:10.910142 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:10.910241 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:10.911805 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.912606 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.914015 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.914612 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.914731 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.917774 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:10.917780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:10.917784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:10.917826 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:10.920694 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:10.921528 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:10.921532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:10.921535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:10.921663 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:10.922129 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:10.922151 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:10.922153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:10.922536 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:10.922797 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:10.923235 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 20:48:10.923414 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:48:10.928212 16780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:48:10.935942 16780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:48:10.973448 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.951415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.5.78.1:39667)
0926 20:48:10.954686 (+ 3271us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:10.954702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:10.954748 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:48:10.954896 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:10.954910 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:10.955726 (+ 816us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:10.958034 (+ 2308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:48:10.959662 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.964675 (+ 5013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.969804 (+ 5129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.971036 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.971191 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.971461 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:10.972800 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0926 20:48:10.973003 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2945,"thread_start_us":2732,"threads_started":1}
I20260926 20:48:10.974129 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.951102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:39667 (local address 127.0.0.1:40950)
0926 20:48:10.952346 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:10.952408 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:10.952689 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:10.958557 (+ 5868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:10.958572 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:10.958630 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:48:10.959316 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.959339 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.965019 (+ 5680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.965032 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.969560 (+ 4528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.969574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.972050 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.972064 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.972491 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.972565 (+ 74us) client_negotiation.cc:769] Sending connection context
0926 20:48:10.973369 (+ 804us) client_negotiation.cc:240] Negotiation successful
0926 20:48:10.973609 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":65,"thread_start_us":622,"threads_started":1}
I20260926 20:48:10.976600 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.952389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.5.78.3:36831)
0926 20:48:10.965829 (+ 13440us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:10.965842 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:10.966972 (+ 1130us) server_negotiation.cc:407] Connection header received
0926 20:48:10.967150 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:10.967163 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:10.967589 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:10.967850 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:48:10.968822 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.970662 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.973681 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.975181 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.975301 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.975538 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:10.976093 (+ 555us) server_negotiation.cc:299] Negotiation successful
0926 20:48:10.976259 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13190,"thread_start_us":12986,"threads_started":1}
I20260926 20:48:10.976586 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.951722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:36831 (local address 127.0.0.1:40056)
0926 20:48:10.966533 (+ 14811us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:10.966567 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:10.966774 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:10.967963 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:10.967969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:10.968002 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:48:10.968572 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.968586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.970850 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.970855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.973419 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.973437 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.975700 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.975707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.975898 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.975933 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:48:10.976033 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 20:48:10.976180 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14581,"mutex_wait_us":46,"spinlock_wait_cycles":17536,"thread_start_us":14319,"threads_started":1}
I20260926 20:48:10.977837 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.954280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:38411 (local address 127.0.0.1:49568)
0926 20:48:10.967971 (+ 13691us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:10.968004 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:10.968198 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:10.969037 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:10.969042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:10.969071 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:48:10.969466 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.969483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.971544 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.971550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.974830 (+ 3280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:10.974850 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.976972 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.976978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:10.977149 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.977174 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:48:10.977291 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 20:48:10.977434 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13524,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":13313,"threads_started":1}
I20260926 20:48:10.977862 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:10.954890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.5.78.2:38411)
0926 20:48:10.964880 (+ 9990us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:10.964895 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:10.968136 (+ 3241us) server_negotiation.cc:407] Connection header received
0926 20:48:10.968450 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:10.968457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:10.968756 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:10.969043 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:48:10.969713 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.971332 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.975272 (+ 3940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:10.976727 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:10.976908 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:10.977156 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:10.977374 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 20:48:10.977559 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9717,"thread_start_us":10039,"threads_started":1}
I20260926 20:48:10.979837 16928 tablet_service.cc:1467] Processing CreateTablet for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 (DEFAULT_TABLE table=test-table [id=5d5715e32d56449d9e0fbd9c0eb53ff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:48:10.981905 16928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:48:10.981660 17071 tablet_service.cc:1467] Processing CreateTablet for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 (DEFAULT_TABLE table=test-table [id=5d5715e32d56449d9e0fbd9c0eb53ff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:48:10.982103 17205 tablet_service.cc:1467] Processing CreateTablet for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 (DEFAULT_TABLE table=test-table [id=5d5715e32d56449d9e0fbd9c0eb53ff3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:48:10.983202 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:48:10.983892 17205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:48:10.997248 17289 tablet_bootstrap.cc:492] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Bootstrap starting.
I20260926 20:48:11.000530 17290 tablet_bootstrap.cc:492] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Bootstrap starting.
I20260926 20:48:11.001959 17289 tablet_bootstrap.cc:654] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Neither blocks nor log segments found. Creating new log.
I20260926 20:48:11.003052 17291 tablet_bootstrap.cc:492] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Bootstrap starting.
I20260926 20:48:11.003552 17289 log.cc:824] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Log is configured to *not* fsync() on all Append() calls
I20260926 20:48:11.004927 17290 tablet_bootstrap.cc:654] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Neither blocks nor log segments found. Creating new log.
I20260926 20:48:11.006379 17290 log.cc:824] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:48:11.008316 17291 tablet_bootstrap.cc:654] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Neither blocks nor log segments found. Creating new log.
I20260926 20:48:11.008749 17289 tablet_bootstrap.cc:492] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: No bootstrap required, opened a new log
I20260926 20:48:11.009085 17289 ts_tablet_manager.cc:1397] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260926 20:48:11.009934 17291 log.cc:824] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Log is configured to *not* fsync() on all Append() calls
I20260926 20:48:11.011842 17290 tablet_bootstrap.cc:492] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: No bootstrap required, opened a new log
I20260926 20:48:11.012151 17290 ts_tablet_manager.cc:1397] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260926 20:48:11.013415 17291 tablet_bootstrap.cc:492] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: No bootstrap required, opened a new log
I20260926 20:48:11.013768 17291 ts_tablet_manager.cc:1397] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260926 20:48:11.020711 17289 raft_consensus.cc:348] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.021497 17289 raft_consensus.cc:374] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:48:11.021636 17289 raft_consensus.cc:729] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e90b1496152492b88fff5fe00574a86, State: Initialized, Role: FOLLOWER
I20260926 20:48:11.022679 17289 consensus_queue.cc:260] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.024530 17136 heartbeater.cc:503] Master 127.5.78.62:46737 was elected leader, sending a full tablet report...
I20260926 20:48:11.025274 17289 ts_tablet_manager.cc:1428] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260926 20:48:11.023881 17290 raft_consensus.cc:348] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.025681 17290 raft_consensus.cc:374] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:48:11.025810 17290 raft_consensus.cc:729] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b041db2c93e4c9bb44fd9653fe070b4, State: Initialized, Role: FOLLOWER
I20260926 20:48:11.025491 17291 raft_consensus.cc:348] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.026125 17291 raft_consensus.cc:374] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:48:11.026273 17291 raft_consensus.cc:729] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3576c5f5d4104af9bdfb1cd9c0f887ae, State: Initialized, Role: FOLLOWER
I20260926 20:48:11.026793 17290 consensus_queue.cc:260] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.027021 17291 consensus_queue.cc:260] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.028833 17290 ts_tablet_manager.cc:1428] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260926 20:48:11.030380 17270 heartbeater.cc:503] Master 127.5.78.62:46737 was elected leader, sending a full tablet report...
I20260926 20:48:11.031409 17291 ts_tablet_manager.cc:1428] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Time spent starting tablet: real 0.017s user 0.014s sys 0.005s
I20260926 20:48:11.080299 17295 raft_consensus.cc:482] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:48:11.080706 17295 raft_consensus.cc:504] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.082885 17295 leader_election.cc:288] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b041db2c93e4c9bb44fd9653fe070b4 (127.5.78.1:39667), 3576c5f5d4104af9bdfb1cd9c0f887ae (127.5.78.3:36831)
I20260926 20:48:11.090343 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.083351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:59399 (local address 127.5.78.3:36831)
0926 20:48:11.083543 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:11.083550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:11.084149 (+ 599us) server_negotiation.cc:407] Connection header received
0926 20:48:11.084296 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:11.084301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:11.084387 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:11.084479 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:48:11.085151 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.086315 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.088283 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.089169 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.089271 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.089352 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:11.089921 (+ 569us) server_negotiation.cc:299] Negotiation successful
0926 20:48:11.090061 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:48:11.090519 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.083115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:36831 (local address 127.5.78.2:59399)
0926 20:48:11.084009 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:11.084036 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:11.084172 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:11.084523 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:11.084527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:11.084551 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:48:11.084987 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:11.084998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.086478 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.086483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:11.088111 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:11.088123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.089432 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.089439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:11.089642 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.089679 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:48:11.089885 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 20:48:11.090040 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":28,"thread_start_us":433,"threads_started":1}
I20260926 20:48:11.090662 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.083333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:41779 (local address 127.5.78.1:39667)
0926 20:48:11.083546 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:11.083552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:11.084345 (+ 793us) server_negotiation.cc:407] Connection header received
0926 20:48:11.084518 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:11.084523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:11.084600 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:11.084703 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:48:11.085791 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.086660 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.088549 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.089503 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.089641 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.089743 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:11.090203 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 20:48:11.090377 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:48:11.090894 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.083294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:39667 (local address 127.5.78.2:41779)
0926 20:48:11.084209 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0926 20:48:11.084227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:48:11.084351 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:48:11.084744 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:48:11.084747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:48:11.084765 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:48:11.085618 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:11.085632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.086812 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.086816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:11.088364 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:48:11.088378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.089776 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.089788 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:48:11.090018 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.090052 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:48:11.090176 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 20:48:11.090315 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":20,"thread_start_us":511,"threads_started":1}
I20260926 20:48:11.091840 17225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab96b5bdeaf4ab39cc1d54936e1ddc2" candidate_uuid: "9e90b1496152492b88fff5fe00574a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" is_pre_election: true
I20260926 20:48:11.091979 16948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab96b5bdeaf4ab39cc1d54936e1ddc2" candidate_uuid: "9e90b1496152492b88fff5fe00574a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" is_pre_election: true
I20260926 20:48:11.092414 17225 raft_consensus.cc:2454] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e90b1496152492b88fff5fe00574a86 in term 0.
I20260926 20:48:11.092520 16948 raft_consensus.cc:2454] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e90b1496152492b88fff5fe00574a86 in term 0.
I20260926 20:48:11.093170 17026 leader_election.cc:302] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3576c5f5d4104af9bdfb1cd9c0f887ae, 9e90b1496152492b88fff5fe00574a86; no voters:
I20260926 20:48:11.093614 17295 raft_consensus.cc:2781] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:48:11.093786 17295 raft_consensus.cc:482] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:48:11.093880 17295 raft_consensus.cc:3037] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:48:11.096271 17295 raft_consensus.cc:504] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.096947 17295 leader_election.cc:288] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [CANDIDATE]: Term 1 election: Requested vote from peers 5b041db2c93e4c9bb44fd9653fe070b4 (127.5.78.1:39667), 3576c5f5d4104af9bdfb1cd9c0f887ae (127.5.78.3:36831)
I20260926 20:48:11.097589 17225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab96b5bdeaf4ab39cc1d54936e1ddc2" candidate_uuid: "9e90b1496152492b88fff5fe00574a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae"
I20260926 20:48:11.097575 16948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab96b5bdeaf4ab39cc1d54936e1ddc2" candidate_uuid: "9e90b1496152492b88fff5fe00574a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b041db2c93e4c9bb44fd9653fe070b4"
I20260926 20:48:11.097836 17225 raft_consensus.cc:3037] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:48:11.097856 16948 raft_consensus.cc:3037] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:48:11.101671 17225 raft_consensus.cc:2454] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e90b1496152492b88fff5fe00574a86 in term 1.
I20260926 20:48:11.101670 16948 raft_consensus.cc:2454] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e90b1496152492b88fff5fe00574a86 in term 1.
I20260926 20:48:11.102263 17026 leader_election.cc:302] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3576c5f5d4104af9bdfb1cd9c0f887ae, 5b041db2c93e4c9bb44fd9653fe070b4, 9e90b1496152492b88fff5fe00574a86; no voters:
I20260926 20:48:11.102734 17295 raft_consensus.cc:2781] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:48:11.103186 17295 raft_consensus.cc:686] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 1 LEADER]: Becoming Leader. State: Replica: 9e90b1496152492b88fff5fe00574a86, State: Running, Role: LEADER
I20260926 20:48:11.103714 17295 consensus_queue.cc:237] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } }
I20260926 20:48:11.109246 16778 catalog_manager.cc:5495] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e90b1496152492b88fff5fe00574a86 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "9e90b1496152492b88fff5fe00574a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e90b1496152492b88fff5fe00574a86" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38411 } health_report { overall_health: HEALTHY } } }
W20260926 20:48:11.122855 17271 tablet.cc:2369] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:48:11.163372 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.157362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.5.78.2:38411)
0926 20:48:11.157621 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:11.157626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:11.157643 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:48:11.157735 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:11.157739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:11.157842 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:11.157987 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:48:11.158491 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.159223 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.160746 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.161192 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.161299 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.161944 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:48:11.162614 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:48:11.162754 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:11.162924 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 20:48:11.163002 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260926 20:48:11.172331 16948 raft_consensus.cc:1261] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 1 FOLLOWER]: Refusing update from remote peer 9e90b1496152492b88fff5fe00574a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:48:11.172327 17225 raft_consensus.cc:1261] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 1 FOLLOWER]: Refusing update from remote peer 9e90b1496152492b88fff5fe00574a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:48:11.173556 17300 consensus_queue.cc:1035] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3576c5f5d4104af9bdfb1cd9c0f887ae" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 36831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:48:11.174031 17295 consensus_queue.cc:1035] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b041db2c93e4c9bb44fd9653fe070b4" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:48:11.208045 16780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:48:11.208367 16780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:48:11.212527 16780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2e4f9de7d034527a864c4c69281dad8: Sending DeleteTablet for 3 replicas of tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2
I20260926 20:48:11.213850 16928 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:48:11 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40950
I20260926 20:48:11.213967 17205 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:48:11 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:40056
I20260926 20:48:11.214737 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:48:11 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:49568
I20260926 20:48:11.215128 17315 tablet_replica.cc:330] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: stopping tablet replica
I20260926 20:48:11.215603 17315 raft_consensus.cc:2229] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:48:11.215962 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 16860
I20260926 20:48:11.216132 17315 raft_consensus.cc:2258] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:48:11.216995 17314 tablet_replica.cc:330] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: stopping tablet replica
I20260926 20:48:11.217478 17314 raft_consensus.cc:2229] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:48:11.217985 17314 raft_consensus.cc:2258] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:48:11.220621 17314 ts_tablet_manager.cc:1905] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:48:11.225912 17316 tablet_replica.cc:330] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: stopping tablet replica
I20260926 20:48:11.226496 17316 raft_consensus.cc:2229] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:48:11.227288 17316 raft_consensus.cc:2258] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:48:11.229187 17316 ts_tablet_manager.cc:1905] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:48:11.229177 17315 ts_tablet_manager.cc:1905] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:48:11.232088 17314 ts_tablet_manager.cc:1918] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:48:11.232331 17314 log.cc:1196] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/4ab96b5bdeaf4ab39cc1d54936e1ddc2
I20260926 20:48:11.232950 17314 ts_tablet_manager.cc:1939] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 5b041db2c93e4c9bb44fd9653fe070b4: Deleting consensus metadata
I20260926 20:48:11.234984 16765 catalog_manager.cc:4841] TS 5b041db2c93e4c9bb44fd9653fe070b4 (127.5.78.1:39667): tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 (table test-table [id=5d5715e32d56449d9e0fbd9c0eb53ff3]) successfully deleted
I20260926 20:48:11.235848 17315 ts_tablet_manager.cc:1918] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:48:11.236032 17315 log.cc:1196] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/4ab96b5bdeaf4ab39cc1d54936e1ddc2
I20260926 20:48:11.236470 17315 ts_tablet_manager.cc:1939] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 3576c5f5d4104af9bdfb1cd9c0f887ae: Deleting consensus metadata
I20260926 20:48:11.236944 17316 ts_tablet_manager.cc:1918] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:48:11.237164 17316 log.cc:1196] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/4ab96b5bdeaf4ab39cc1d54936e1ddc2
Sep 26 20:48:11 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:48:11.237783 17316 ts_tablet_manager.cc:1939] T 4ab96b5bdeaf4ab39cc1d54936e1ddc2 P 9e90b1496152492b88fff5fe00574a86: Deleting consensus metadata
I20260926 20:48:11.238329 16766 catalog_manager.cc:4841] TS 3576c5f5d4104af9bdfb1cd9c0f887ae (127.5.78.3:36831): tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 (table test-table [id=5d5715e32d56449d9e0fbd9c0eb53ff3]) successfully deleted
I20260926 20:48:11.239528 16764 catalog_manager.cc:4841] TS 9e90b1496152492b88fff5fe00574a86 (127.5.78.2:38411): tablet 4ab96b5bdeaf4ab39cc1d54936e1ddc2 (table test-table [id=5d5715e32d56449d9e0fbd9c0eb53ff3]) successfully deleted
I20260926 20:48:11.246500 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.218029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40952 (local address 127.5.78.1:39667)
0926 20:48:11.218275 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:11.218285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:11.226572 (+ 8287us) server_negotiation.cc:407] Connection header received
0926 20:48:11.226836 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:11.226843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:11.226937 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:11.227024 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:11.228992 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.230042 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.232031 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.232738 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.232839 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.239640 (+ 6801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:11.239662 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:11.239680 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:11.239746 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:11.242974 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:11.243773 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:11.243782 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:11.243785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:11.243939 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:11.244422 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:11.244429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:11.244434 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:11.244916 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:11.245201 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:11.245797 (+ 596us) server_negotiation.cc:299] Negotiation successful
0926 20:48:11.246041 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:48:11.247924 16988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:48:11.357496 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 16860
I20260926 20:48:11.384305 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 16999
Sep 26 20:48:11 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:48:11.405174 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.385239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.5.78.2:38411)
0926 20:48:11.385411 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:11.385416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:11.385441 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:48:11.385696 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:11.385700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:11.385763 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:11.385908 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:11.387811 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.389166 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.391279 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.391999 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.392151 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.397422 (+ 5271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:11.397440 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:11.397456 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:11.397499 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:11.401591 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:11.402763 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:11.402774 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:11.402778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:11.402967 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:11.403395 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:11.403400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:11.403404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:11.403878 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:11.404231 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:11.404513 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 20:48:11.404753 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:48:11.405989 17131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:48:11.523352 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 16999
I20260926 20:48:11.541864 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 17142
Sep 26 20:48:11 dist-test-slave-z7j8 krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:48:11.558126 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:11.542701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.5.78.3:36831)
0926 20:48:11.542921 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:11.542928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:11.543038 (+ 110us) server_negotiation.cc:407] Connection header received
0926 20:48:11.543280 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:11.543291 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:11.543357 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:11.543469 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:11.544780 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.545530 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.546986 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:11.547371 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:11.547518 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:11.552215 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:11.552232 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:11.552249 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:11.552292 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:11.555494 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:11.556262 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:11.556270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:11.556272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:11.556401 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:11.556779 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:11.556782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:11.556784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:11.557128 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:11.557350 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:11.557701 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 20:48:11.557864 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:48:11.558851 17265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:48:11.648223 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 17142
I20260926 20:48:11.664789 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 16748
I20260926 20:48:11.666226 16810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:48:11.780297 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 16748
I20260926 20:48:11.802227 5432 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 20:48:12.138778 5432 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 20:48:12.138989 5432 mini_ranger.cc:66] Stopping Ranger...
I20260926 20:48:12.480422 5432 mini_ranger.cc:68] Stopped Ranger
2026-09-26 20:48:12.480 UTC [16334] LOG: received smart shutdown request
2026-09-26 20:48:12.484 UTC [16334] LOG: background worker "logical replication launcher" (PID 16343) exited with exit code 1
2026-09-26 20:48:12.485 UTC [16338] LOG: shutting down
2026-09-26 20:48:12.704 UTC [16334] LOG: database system is shut down
2026-09-26T20:48:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50832 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:48:12 dist-test-slave-z7j8 krb5kdc[17330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:48:12 dist-test-slave-z7j8 krb5kdc[17330](info): set up 2 sockets
Sep 26 20:48:12 dist-test-slave-z7j8 krb5kdc[17330](info): commencing operation
krb5kdc: starting...
W20260926 20:48:13.962905 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.010s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:48:14 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:48:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/rangeradmin_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/rangeradmin_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/rangerlookup_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/rangerlookup_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
I20260926 20:48:14.310305 5432 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskb6fvOW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/postgres -l logfile start
2026-09-26 20:48:17.261 UTC [17360] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 20:48:17.261 UTC [17360] LOG: listening on IPv4 address "127.5.78.20", port 46819
2026-09-26 20:48:17.265 UTC [17360] LOG: listening on Unix socket "/tmp/.s.PGSQL.46819"
2026-09-26 20:48:17.290 UTC [17363] LOG: database system was shut down at 2026-09-26 20:48:15 UTC
2026-09-26 20:48:17.295 UTC [17360] LOG: database system is ready to accept connections
I20260926 20:48:18.262346 5432 mini_postgres.cc:96] Postgres bound to 46819
2026-09-26 20:48:18.279 UTC [17371] FATAL: database "slave" does not exist
127.5.78.20:46819 - accepting connections
I20260926 20:48:18.280405 5432 mini_ranger.cc:161] Starting Ranger...
I20260926 20:48:18.307971 5432 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 20:48:19.107313 5432 mini_ranger.cc:87] Created ranger Postgres database
I20260926 20:48:19.107438 5432 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-admin
2026-09-26 20:48:19,420 [I] DB FLAVOR :POSTGRES
2026-09-26 20:48:19,421 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:48:19,421 [I] Checking connection..
2026-09-26 20:48:19,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 20:48:19,701 [I] Checking connection passed.
2026-09-26 20:48:19,701 [I] --------- Verifying version history table ---------
2026-09-26 20:48:19,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:48:19,950 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:48:19,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:48:20,204 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:48:20,204 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 20:48:20,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 20:48:20.445 UTC [17435] WARNING: there is no transaction in progress
2026-09-26 20:48:20,453 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 20:48:20,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:48:20,740 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 20:48:20,740 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:48:20,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 20:48:21,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 20:48:21,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z7j8.c.gcp-upstream.internal','N') ;"
2026-09-26 20:48:21,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:48:21,810 [I] Table x_portal_user does not exist in database ranger
2026-09-26 20:48:21,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:48:22,052 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 20:48:22,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:48:22,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:48:22,537 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 20:48:22,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 20:48:23.611 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.629 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.646 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.663 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.684 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.703 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.763 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.771 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.788 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.800 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.811 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.826 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.837 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.844 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.852 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:23.860 UTC [17543] WARNING: there is no transaction in progress
2026-09-26 20:48:24,612 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 20:48:24,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 20:48:24,857 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 20:48:24,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:48:25,105 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 20:48:25,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:48:25,358 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 20:48:25,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:48:25,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:48:25,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z7j8.c.gcp-upstream.internal';"
2026-09-26 20:48:26,126 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 20:48:26,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 20:48:26,364 [I] DB_PATCHES have already been applied
I20260926 20:48:26.370707 5432 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-admin:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskb6fvOW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 20:48:26.370793 5432 mini_ranger.cc:193] Using host: 127.5.78.20
I20260926 20:48:26.386533 5432 mini_ranger.cc:227] Ranger admin URL: http://127.5.78.20:40905
Sep 26, 2026 8:48:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:48:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:48:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 8:48:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 8:48:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 8:48:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 8:48:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 20:48:27.597190 5432 mini_ranger.cc:160] Time spent starting Ranger: real 9.317s user 0.010s sys 0.032s
Sep 26, 2026 8:48:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 8:48:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 8:48:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 8:48:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 8:48:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 20:48:49.310819 5432 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.78.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.78.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/rangerkms_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.78.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/rangerkms_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.78.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.78.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.78.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/HTTP_127.5.78.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 20:48:49 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455729, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 20:48:49.475515 5432 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 20:48:49.498032 5432 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 20:48:50.557842 5432 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 20:48:50.557978 5432 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms
I20260926 20:48:50.558017 5432 mini_ranger_kms.cc:227] Using postgres at 127.5.78.20:46819
2026-09-26 20:48:50,949 [I] DB FLAVOR :POSTGRES
2026-09-26 20:48:50,950 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:48:50,950 [I] Checking connection
2026-09-26 20:48:50,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 20:48:51,216 [I] connection success
2026-09-26 20:48:51,216 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 20:48:51,216 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 20:48:51,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 20:48:51,495 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 20:48:51,495 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:48:51,495 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 20:48:51,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.78.20:46819/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 20:48:51,840 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 20:48:52.353751 5432 mini_ranger_kms.cc:310] Created kms service
I20260926 20:48:52.568938 5432 mini_ranger_kms.cc:326] Created kudu user
I20260926 20:48:52.616804 5432 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 20:48:52 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455729, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:53.054498 5432 mini_ranger_kms.cc:384] Added ranger policy
I20260926 20:48:53.054859 5432 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms:/tmp/dist-test-taskb6fvOW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskb6fvOW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 20:48:53.054921 5432 mini_ranger_kms.cc:240] Using host: 127.5.78.20
I20260926 20:48:53.075472 5432 mini_ranger_kms.cc:276] Ranger KMS PID: 17724
I20260926 20:48:53.075604 5432 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.78.20:35107
Sep 26, 2026 8:48:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:48:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:48:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 8:48:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 8:48:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 8:48:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 20:48:55.374502 5432 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.899s user 0.092s sys 0.313s
I20260926 20:48:55.374776 5432 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 20:48:55.374955 5432 mini_ranger_kms.cc:401] 127.5.78.20:35107/kms/v1/keys
Sep 26 20:48:55 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455735, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.78.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 20:48:55 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455735, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.78.20@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
Sep 26 20:48:56 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455735, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.78.20@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
Sep 26 20:48:58 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:48:58.794629 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:37875
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:37875
--ranger_config_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:48:59.147682 17774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:48:59.148087 17774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:48:59.148262 17774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:48:59.155944 17774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:48:59.156105 17774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:48:59.156212 17774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:48:59.156314 17774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:48:59.156400 17774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:48:59.166747 17774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.78.20:35107/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:37875
--ranger_config_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:48:59.168146 17774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:48:59.169553 17774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:48:59.182240 17779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:59.182283 17780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:48:59.182322 17782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:48:59.182857 17774 server_base.cc:1034] running on GCE node
I20260926 20:48:59.183764 17774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:48:59.185424 17774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:48:59.186663 17774 hybrid_clock.cc:648] HybridClock initialized: now 1790455739186614 us; error 55 us; skew 500 ppm
I20260926 20:48:59.187059 17774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:48:59 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455739, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:48:59.197723 17774 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:48:59.199472 17774 webserver.cc:460] Webserver started at http://127.5.78.62:33751/ using document root <none> and password file <none>
I20260926 20:48:59.200196 17774 fs_manager.cc:359] Metadata directory not provided
I20260926 20:48:59.200338 17774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:48:59.200642 17774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:48:59 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455739, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:48:59.253325 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "c45cc4addc8d4a7d8aea9affde069bc3"
format_stamp: "Formatted at 2026-09-26 20:48:59 on dist-test-slave-z7j8"
server_key: "fdc2c38d7f2ecbc5029957a95350ff4b"
server_key_iv: "21804d3b783dfccfae086953ed0fec38"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:59.254482 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "c45cc4addc8d4a7d8aea9affde069bc3"
format_stamp: "Formatted at 2026-09-26 20:48:59 on dist-test-slave-z7j8"
server_key: "fdc2c38d7f2ecbc5029957a95350ff4b"
server_key_iv: "21804d3b783dfccfae086953ed0fec38"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:59.261772 17774 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260926 20:48:59.283408 17789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:48:59.285315 17774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:48:59.285547 17774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c45cc4addc8d4a7d8aea9affde069bc3"
format_stamp: "Formatted at 2026-09-26 20:48:59 on dist-test-slave-z7j8"
server_key: "fdc2c38d7f2ecbc5029957a95350ff4b"
server_key_iv: "21804d3b783dfccfae086953ed0fec38"
server_key_version: "kuduclusterkey@0"
I20260926 20:48:59.285821 17774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:48:59.315006 17774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:48:59.319916 17774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:48:59.320194 17774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:48:59.339392 17774 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:37875
I20260926 20:48:59.339394 17840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:37875 every 8 connection(s)
I20260926 20:48:59.340898 17774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:48:59.342419 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 17774
I20260926 20:48:59.342633 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:48:59.343525 5432 external_mini_cluster.cc:1428] Setting key d7e8e9a75504e1ef28b37d83797ad561
I20260926 20:48:59.345995 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:48:59 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:48:59.357863 17841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Bootstrap starting.
I20260926 20:48:59.361963 17841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Neither blocks nor log segments found. Creating new log.
I20260926 20:48:59.363229 17841 log.cc:824] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Log is configured to *not* fsync() on all Append() calls
I20260926 20:48:59.366628 17841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: No bootstrap required, opened a new log
I20260926 20:48:59.366959 17844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:48:59.345971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54318 (local address 127.5.78.62:37875)
0926 20:48:59.346945 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0926 20:48:59.346959 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:48:59.347006 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:48:59.348698 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:48:59.348721 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:48:59.349181 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:48:59.349634 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:48:59.351069 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:59.352410 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:59.353706 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:48:59.354384 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:48:59.354622 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:48:59.358840 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:48:59.358851 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:48:59.358867 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:48:59.358913 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:48:59.362742 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:59.363365 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:59.363373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:59.363380 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:59.363564 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:48:59.364033 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:48:59.364036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:48:59.364038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:48:59.364571 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:48:59.364849 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 20:48:59.365243 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 20:48:59.365635 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":496,"threads_started":1}
I20260926 20:48:59.374939 17841 raft_consensus.cc:348] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:48:59.375361 17841 raft_consensus.cc:374] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:48:59.375473 17841 raft_consensus.cc:729] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45cc4addc8d4a7d8aea9affde069bc3, State: Initialized, Role: FOLLOWER
I20260926 20:48:59.376132 17841 consensus_queue.cc:260] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:48:59.376405 17841 raft_consensus.cc:388] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:48:59.376566 17841 raft_consensus.cc:482] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:48:59.376746 17841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:48:59.378970 17841 raft_consensus.cc:504] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:48:59.379485 17841 leader_election.cc:302] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c45cc4addc8d4a7d8aea9affde069bc3; no voters:
I20260926 20:48:59.380038 17841 leader_election.cc:288] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:48:59.380174 17846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:48:59.380891 17846 raft_consensus.cc:686] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 LEADER]: Becoming Leader. State: Replica: c45cc4addc8d4a7d8aea9affde069bc3, State: Running, Role: LEADER
I20260926 20:48:59.381554 17846 consensus_queue.cc:237] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:48:59.382150 17841 sys_catalog.cc:564] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:48:59.384312 17847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } } }
I20260926 20:48:59.384761 17847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: This master's current role is: LEADER
I20260926 20:48:59.384740 17848 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c45cc4addc8d4a7d8aea9affde069bc3. Latest consensus state: current_term: 1 leader_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } } }
I20260926 20:48:59.385113 17848 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: This master's current role is: LEADER
I20260926 20:48:59.388403 17854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:48:59.393832 17854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:48:59.394505 17841 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 20:48:59.411473 17854 catalog_manager.cc:1347] Generated new cluster ID: 9df1e9a388a64f26803076dc1069f6b9
I20260926 20:48:59.411643 17854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:48:59.447175 17854 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:48:59.449232 17854 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:48:59.468592 17854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Generated new TSK 0
I20260926 20:48:59.469806 17854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:49:00.234972 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:37875
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:49:00 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 20:49:00.675678 17886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:00.676033 17886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:00.676196 17886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:00.684158 17886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:00.684312 17886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:00.684548 17886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:49:00.696700 17886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.78.20:35107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:00.698377 17886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:00.700258 17886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:00.727555 17896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:00.728650 17897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:00.746162 17899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:00.746536 17886 server_base.cc:1034] running on GCE node
I20260926 20:49:00.747519 17886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:00.748783 17886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:00.750070 17886 hybrid_clock.cc:648] HybridClock initialized: now 1790455740750000 us; error 68 us; skew 500 ppm
I20260926 20:49:00.750465 17886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:00 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:00.753981 17886 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:49:00.760310 17886 webserver.cc:460] Webserver started at http://127.5.78.1:36283/ using document root <none> and password file <none>
I20260926 20:49:00.761111 17886 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:00.761258 17886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:00.761593 17886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:49:00 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:00.781229 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "7146febe7ff14bc0969cf39fa17f13fa"
format_stamp: "Formatted at 2026-09-26 20:49:00 on dist-test-slave-z7j8"
server_key: "bdb8a17b7d41146a0b264fceb0d392d8"
server_key_iv: "3cd2f34a8b042ba5e4a68a56f3bbac15"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:00.782603 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "7146febe7ff14bc0969cf39fa17f13fa"
format_stamp: "Formatted at 2026-09-26 20:49:00 on dist-test-slave-z7j8"
server_key: "bdb8a17b7d41146a0b264fceb0d392d8"
server_key_iv: "3cd2f34a8b042ba5e4a68a56f3bbac15"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:00.803777 17886 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.012s sys 0.003s
I20260926 20:49:00.827369 17906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:00.829376 17886 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:49:00.829583 17886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "7146febe7ff14bc0969cf39fa17f13fa"
format_stamp: "Formatted at 2026-09-26 20:49:00 on dist-test-slave-z7j8"
server_key: "bdb8a17b7d41146a0b264fceb0d392d8"
server_key_iv: "3cd2f34a8b042ba5e4a68a56f3bbac15"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:00.829843 17886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:00.860561 17886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:00.866453 17886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:00.866869 17886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:00.881407 17886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:00.898233 17886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:00.898495 17886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 20:49:00.898731 17886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:00.898878 17886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:49:01 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:01.312080 17886 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:34447
I20260926 20:49:01.312110 18018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:34447 every 8 connection(s)
I20260926 20:49:01.313865 17886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:49:01.323684 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 17886
I20260926 20:49:01.323931 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:49:01.324838 5432 external_mini_cluster.cc:1428] Setting key 97928b51576b3e40210c65e49af9b8f2
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
Sep 26 20:49:01 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:49:01.392935 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:01.317152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:39891 (local address 127.5.78.62:37875)
0926 20:49:01.327492 (+ 10340us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:01.327500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:01.340921 (+ 13421us) server_negotiation.cc:407] Connection header received
0926 20:49:01.350478 (+ 9557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:01.350485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:01.350589 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:01.350725 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:01.358610 (+ 7885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:01.360055 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:01.366613 (+ 6558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:01.367383 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:01.367521 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:01.383501 (+ 15980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:01.383511 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:01.383516 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:01.383570 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:01.387630 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:01.388866 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:01.388873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:01.388876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:01.389093 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:01.389745 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:01.389751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:01.389754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:01.390143 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:01.390317 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:01.392290 (+ 1973us) server_negotiation.cc:299] Negotiation successful
0926 20:49:01.392545 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10193,"thread_start_us":13298,"threads_started":1}
I20260926 20:49:01.395128 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:01.318078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:37875 (local address 127.5.78.1:39891)
0926 20:49:01.336881 (+ 18803us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:01.336939 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:01.349938 (+ 12999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:01.354376 (+ 4438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:01.354392 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:01.355158 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:01.356114 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:01.356153 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:01.360379 (+ 4226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:01.360386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:01.363050 (+ 2664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:01.363082 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:01.367949 (+ 4867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:01.367959 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:01.368344 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:01.369367 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:01.369403 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:01.383062 (+ 13659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:01.387892 (+ 4830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:01.387904 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:01.387926 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:01.388670 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:01.389267 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:01.389272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:01.389276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:01.389577 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:01.390455 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:01.390468 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:01.391029 (+ 561us) client_negotiation.cc:769] Sending connection context
0926 20:49:01.391394 (+ 365us) client_negotiation.cc:240] Negotiation successful
0926 20:49:01.392010 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17318,"thread_start_us":17289,"threads_started":1}
I20260926 20:49:01.397692 18019 heartbeater.cc:347] Connected to a master server at 127.5.78.62:37875
I20260926 20:49:01.398047 18019 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:01.398947 18019 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:01.401496 17806 ts_manager.cc:194] Registered new tserver with Master: 7146febe7ff14bc0969cf39fa17f13fa (127.5.78.1:34447)
I20260926 20:49:01.403507 17806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:39891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:49:01.424861 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:37875
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:49:01 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
W20260926 20:49:01.949507 18025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:01.949898 18025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:01.950052 18025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:01.958005 18025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:01.958184 18025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:01.958369 18025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:49:01.969188 18025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.78.20:35107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:01.970710 18025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:01.972146 18025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:01.988396 18039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:01.988338 18040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:01.988338 18042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:01.990346 18025 server_base.cc:1034] running on GCE node
I20260926 20:49:01.991185 18025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:01.992384 18025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:01.993649 18025 hybrid_clock.cc:648] HybridClock initialized: now 1790455741993571 us; error 68 us; skew 500 ppm
I20260926 20:49:01.994050 18025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:01 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:01.997503 18025 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:49:01.999375 18025 webserver.cc:460] Webserver started at http://127.5.78.2:39277/ using document root <none> and password file <none>
I20260926 20:49:02.000154 18025 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:02.000300 18025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:02.000687 18025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:49:01 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:02.019423 18025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "7633870685e64abe87558d507e8e4c33"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "55c5b669dcf2f7b7b76f2601fcc621d1"
server_key_iv: "5c343bd4e101a4df4b0a580e6f256b31"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:02.020637 18025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "7633870685e64abe87558d507e8e4c33"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "55c5b669dcf2f7b7b76f2601fcc621d1"
server_key_iv: "5c343bd4e101a4df4b0a580e6f256b31"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:02.028283 18025 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260926 20:49:02.053989 18049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:02.056180 18025 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.009s sys 0.000s
I20260926 20:49:02.056392 18025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "7633870685e64abe87558d507e8e4c33"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "55c5b669dcf2f7b7b76f2601fcc621d1"
server_key_iv: "5c343bd4e101a4df4b0a580e6f256b31"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:02.056660 18025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:02.078094 18025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:02.083484 18025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:02.083868 18025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:02.085018 18025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:02.091354 18025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:02.091473 18025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:02.091645 18025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:02.091710 18025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:02.130038 18025 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:34865
I20260926 20:49:02.130075 18161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:34865 every 8 connection(s)
I20260926 20:49:02.131814 18025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:49:02.138033 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18025
I20260926 20:49:02.138216 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:49:02.138827 5432 external_mini_cluster.cc:1428] Setting key 7fef9c43f6d8dd9d9d450c2bd6ec0bfb
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:02.154942 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.134568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:34645 (local address 127.5.78.62:37875)
0926 20:49:02.135091 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:02.135098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:02.136194 (+ 1096us) server_negotiation.cc:407] Connection header received
0926 20:49:02.138879 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:02.138885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:02.138987 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:02.139114 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:02.140907 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.141621 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.143285 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.143707 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.143843 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.147436 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:02.147441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:02.147445 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:02.147482 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:02.150883 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.151988 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.151994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.151998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.152184 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.152772 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.152777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.152780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.153134 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:02.153276 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:02.154275 (+ 999us) server_negotiation.cc:299] Negotiation successful
0926 20:49:02.154579 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":245,"threads_started":1}
I20260926 20:49:02.156342 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.135154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:37875 (local address 127.5.78.2:34645)
0926 20:49:02.136005 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:02.136055 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:02.138506 (+ 2451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:02.139419 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:02.139433 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:02.140047 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:02.140687 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:02.140713 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.141782 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.141787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:02.143131 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:02.143150 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.143933 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.143941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:02.144195 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.144909 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:02.144937 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:02.147160 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:02.151054 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:02.151067 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:02.151092 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:02.151805 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:02.152311 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:02.152318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:02.152325 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:02.152636 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:02.153274 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:02.153285 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:02.153825 (+ 540us) client_negotiation.cc:769] Sending connection context
0926 20:49:02.154142 (+ 317us) client_negotiation.cc:240] Negotiation successful
0926 20:49:02.154796 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":367,"threads_started":1}
I20260926 20:49:02.158120 18162 heartbeater.cc:347] Connected to a master server at 127.5.78.62:37875
I20260926 20:49:02.158521 18162 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:02.159454 18162 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:02.161403 17806 ts_manager.cc:194] Registered new tserver with Master: 7633870685e64abe87558d507e8e4c33 (127.5.78.2:34865)
I20260926 20:49:02.162909 17806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:34645
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:49:02.198020 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:37875
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:49:02.407614 18019 heartbeater.cc:503] Master 127.5.78.62:37875 was elected leader, sending a full tablet report...
W20260926 20:49:02.537953 18168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:02.538291 18168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:02.538504 18168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:02.545926 18168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:02.546061 18168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:02.546252 18168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:49:02.556306 18168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.78.20:35107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:02.557690 18168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:02.559154 18168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:02.573158 18173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:02.573122 18174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:02.573195 18176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:02.574076 18168 server_base.cc:1034] running on GCE node
I20260926 20:49:02.574805 18168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:02.575773 18168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:02.577069 18168 hybrid_clock.cc:648] HybridClock initialized: now 1790455742577011 us; error 48 us; skew 500 ppm
I20260926 20:49:02.577450 18168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455742, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:02.580883 18168 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:49:02.582708 18168 webserver.cc:460] Webserver started at http://127.5.78.3:44017/ using document root <none> and password file <none>
I20260926 20:49:02.583554 18168 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:02.583706 18168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:02.584026 18168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455742, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:02.599759 18168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "f3441e7d99884bde8e1696c91bce5232"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "9b007c22c8f2b5ef8f699b75ae6347af"
server_key_iv: "b1b809ede7a72e4cf9a35ed7d7a92c70"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:02.600764 18168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "f3441e7d99884bde8e1696c91bce5232"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "9b007c22c8f2b5ef8f699b75ae6347af"
server_key_iv: "b1b809ede7a72e4cf9a35ed7d7a92c70"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:02.606912 18168 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 20:49:02.625537 18183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:02.627252 18168 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:02.627447 18168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "f3441e7d99884bde8e1696c91bce5232"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "9b007c22c8f2b5ef8f699b75ae6347af"
server_key_iv: "b1b809ede7a72e4cf9a35ed7d7a92c70"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:02.627665 18168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:02.648448 18168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:02.652731 18168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:02.653059 18168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:02.654064 18168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:02.655505 18168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:02.655627 18168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:02.655762 18168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:02.655834 18168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:02.689898 18168 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:43675
I20260926 20:49:02.689936 18295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:43675 every 8 connection(s)
I20260926 20:49:02.691381 18168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:49:02.692203 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18168
I20260926 20:49:02.692415 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:49:02.692971 5432 external_mini_cluster.cc:1428] Setting key b12a5608e2d89fc5a543b15f84496d85
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455742, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:02.713670 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.694502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:53523 (local address 127.5.78.62:37875)
0926 20:49:02.695097 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:02.695103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:02.697012 (+ 1909us) server_negotiation.cc:407] Connection header received
0926 20:49:02.698623 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:02.698628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:02.698721 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:02.698841 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:02.700317 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.701029 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.702700 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.703117 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.703269 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.706888 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:02.706893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:02.706897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:02.706946 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:02.710181 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.711215 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.711220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.711223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.711384 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.711939 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.711943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.711945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.712264 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:02.712385 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:02.713164 (+ 779us) server_negotiation.cc:299] Negotiation successful
0926 20:49:02.713378 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":326,"threads_started":1}
I20260926 20:49:02.714656 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.695607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:37875 (local address 127.5.78.3:53523)
0926 20:49:02.696477 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:02.696518 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:02.698297 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:02.699096 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:02.699105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:02.699541 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:02.700143 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:02.700160 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.701204 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.701208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:02.702538 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:02.702558 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.703367 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.703374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:02.703624 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.704334 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:02.704357 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:02.706620 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:02.710384 (+ 3764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:02.710393 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:02.710480 (+ 87us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:02.711022 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:02.711524 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:02.711528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:02.711530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:02.711786 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:02.712390 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:02.712397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:02.712780 (+ 383us) client_negotiation.cc:769] Sending connection context
0926 20:49:02.713048 (+ 268us) client_negotiation.cc:240] Negotiation successful
0926 20:49:02.713462 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":360,"threads_started":1}
I20260926 20:49:02.716218 18296 heartbeater.cc:347] Connected to a master server at 127.5.78.62:37875
I20260926 20:49:02.716958 18296 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:02.717921 18296 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:02.719933 17806 ts_manager.cc:194] Registered new tserver with Master: f3441e7d99884bde8e1696c91bce5232 (127.5.78.3:43675)
I20260926 20:49:02.721154 17806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:53523
I20260926 20:49:02.722328 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:49:02.724041 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 17886
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:49:02.741539 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.725072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.5.78.1:34447)
0926 20:49:02.725747 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:02.725754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:02.725779 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:49:02.725835 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:02.725841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:02.726040 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:02.726171 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:02.727966 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.728988 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.730552 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.731130 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.731240 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.735932 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:02.735944 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:02.735959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:02.735998 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:02.738990 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.739695 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.739704 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.739707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.739843 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.740170 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.740173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.740175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.740499 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:02.740751 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:02.740944 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 20:49:02.741146 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":337,"threads_started":1}
I20260926 20:49:02.743227 18014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:02.826496 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 17886
I20260926 20:49:02.842303 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18025
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:49:02.860347 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.843224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.5.78.2:34865)
0926 20:49:02.843762 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:02.843769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:02.843796 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:49:02.843863 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:02.843870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:02.844076 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:02.844238 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:02.845629 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.846747 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.848257 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.848879 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.848999 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.854092 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:02.854107 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:02.854128 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:02.854171 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:02.857165 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.858016 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.858025 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.858027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.858176 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.858709 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.858714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.858716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.859114 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:02.859361 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:02.859719 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 20:49:02.859956 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":286,"threads_started":1}
I20260926 20:49:02.861778 18157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:02.944279 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18025
I20260926 20:49:02.964498 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18168
Sep 26 20:49:02 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:49:02.983000 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:02.965688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.5.78.3:43675)
0926 20:49:02.966388 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:02.966395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:02.966491 (+ 96us) server_negotiation.cc:407] Connection header received
0926 20:49:02.966557 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:02.966564 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:02.966822 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:02.966952 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:02.968520 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.969555 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.971003 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:02.971609 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:02.971723 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:02.976197 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:02.976213 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:02.976229 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:02.976284 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:02.979445 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.980311 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.980320 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.980323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.980493 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:02.981141 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:02.981146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:02.981148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:02.981562 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:02.981844 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:02.982309 (+ 465us) server_negotiation.cc:299] Negotiation successful
0926 20:49:02.982591 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":389,"threads_started":1}
I20260926 20:49:02.984513 18291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:03.069906 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18168
I20260926 20:49:03.086321 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 17774
I20260926 20:49:03.087754 17836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:03.195426 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 17774
I20260926 20:49:03.218291 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:37875
--webserver_interface=127.5.78.62
--webserver_port=33751
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:37875
--ranger_config_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:03.560700 18308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:03.561048 18308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:03.561215 18308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:03.569983 18308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:03.570128 18308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:03.570220 18308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:03.570305 18308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:03.570438 18308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:03.581202 18308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:37875
--ranger_config_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:03.582736 18308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:03.584275 18308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:03.597349 18314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:03.597309 18313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:03.597389 18316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:03.598714 18308 server_base.cc:1034] running on GCE node
I20260926 20:49:03.599611 18308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:03.601356 18308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:03.602617 18308 hybrid_clock.cc:648] HybridClock initialized: now 1790455743602563 us; error 55 us; skew 500 ppm
I20260926 20:49:03.603046 18308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:03 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455743, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:03.606585 18308 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:03.608340 18308 webserver.cc:460] Webserver started at http://127.5.78.62:33751/ using document root <none> and password file <none>
I20260926 20:49:03.609102 18308 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:03.609256 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:49:03 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455743, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:03.660858 18308 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:49:03.664574 18324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:03.666610 18308 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:03.666851 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c45cc4addc8d4a7d8aea9affde069bc3"
format_stamp: "Formatted at 2026-09-26 20:48:59 on dist-test-slave-z7j8"
server_key: "fdc2c38d7f2ecbc5029957a95350ff4b"
server_key_iv: "21804d3b783dfccfae086953ed0fec38"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:03.668043 18308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:03.697140 18308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:03.701597 18308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:03.702013 18308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:03.722679 18308 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:37875
I20260926 20:49:03.722689 18375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:37875 every 8 connection(s)
I20260926 20:49:03.725289 18308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:03.733245 18376 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:49:03.733306 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18308
I20260926 20:49:03.734354 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:34447
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:37875
--webserver_port=36283
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:49:03.739176 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Bootstrap starting.
I20260926 20:49:03.760299 18376 log.cc:824] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:03.769778 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:49:03.770362 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Bootstrap complete.
I20260926 20:49:03.779944 18376 raft_consensus.cc:348] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:49:03.780862 18376 raft_consensus.cc:729] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45cc4addc8d4a7d8aea9affde069bc3, State: Initialized, Role: FOLLOWER
I20260926 20:49:03.781620 18376 consensus_queue.cc:260] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:49:03.781862 18376 raft_consensus.cc:388] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:49:03.781985 18376 raft_consensus.cc:482] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:49:03.782164 18376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:49:03.786219 18376 raft_consensus.cc:504] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:49:03.786851 18376 leader_election.cc:302] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c45cc4addc8d4a7d8aea9affde069bc3; no voters:
I20260926 20:49:03.787518 18376 leader_election.cc:288] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:49:03.787662 18380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:49:03.788895 18380 raft_consensus.cc:686] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [term 2 LEADER]: Becoming Leader. State: Replica: c45cc4addc8d4a7d8aea9affde069bc3, State: Running, Role: LEADER
I20260926 20:49:03.789525 18380 consensus_queue.cc:237] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } }
I20260926 20:49:03.789983 18376 sys_catalog.cc:564] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:03.793227 18382 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c45cc4addc8d4a7d8aea9affde069bc3. Latest consensus state: current_term: 2 leader_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } } }
I20260926 20:49:03.793758 18382 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:03.794360 18381 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45cc4addc8d4a7d8aea9affde069bc3" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37875 } } }
I20260926 20:49:03.794809 18381 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:03.797264 18386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:03.804615 18386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:49:03.806533 18386 catalog_manager.cc:1259] Loaded cluster ID: 9df1e9a388a64f26803076dc1069f6b9
I20260926 20:49:03.806682 18386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:49:03.812822 18386 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:49:03.815187 18386 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Loaded TSK: 0
I20260926 20:49:03.817427 18386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:49:03.821225 18376 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260926 20:49:04.214231 18378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:04.214725 18378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:04.214885 18378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:04.227650 18378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:04.227814 18378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:04.228018 18378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:49:04.244812 18378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:34447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=36283
--tserver_master_addrs=127.5.78.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:04.246716 18378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:04.248531 18378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:04.283232 18414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:04.286221 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:04.286382 18417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:04.287716 18378 server_base.cc:1034] running on GCE node
I20260926 20:49:04.288661 18378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:04.291075 18378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:04.292781 18378 hybrid_clock.cc:648] HybridClock initialized: now 1790455744292708 us; error 78 us; skew 500 ppm
I20260926 20:49:04.293221 18378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:04 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:04.302799 18378 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:49:04.318209 18378 webserver.cc:460] Webserver started at http://127.5.78.1:36283/ using document root <none> and password file <none>
I20260926 20:49:04.319377 18378 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:04.319607 18378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:49:04 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:04.368799 18378 fs_manager.cc:709] Time spent opening directory manager: real 0.023s user 0.015s sys 0.007s
I20260926 20:49:04.383193 18425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:04.386714 18378 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.004s sys 0.008s
I20260926 20:49:04.389014 18378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "7146febe7ff14bc0969cf39fa17f13fa"
format_stamp: "Formatted at 2026-09-26 20:49:00 on dist-test-slave-z7j8"
server_key: "bdb8a17b7d41146a0b264fceb0d392d8"
server_key_iv: "3cd2f34a8b042ba5e4a68a56f3bbac15"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:04.390662 18378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:04.423478 18378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:04.428344 18378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:04.428654 18378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:04.429853 18378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:04.431385 18378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:04.431504 18378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:04.431612 18378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:04.431708 18378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:04.481102 18378 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:34447
I20260926 20:49:04.481169 18537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:34447 every 8 connection(s)
I20260926 20:49:04.483098 18378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:49:04.488109 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18378
I20260926 20:49:04.488929 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:34865
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:37875
--webserver_port=39277
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:49:04 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455744, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:04.545676 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:04.497761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:37875 (local address 127.5.78.1:35109)
0926 20:49:04.511632 (+ 13871us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:04.511724 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:04.517536 (+ 5812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:04.520531 (+ 2995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:04.520544 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:04.521221 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:04.522091 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:04.522121 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:04.524621 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:04.524627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:04.526742 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:04.526762 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:04.528679 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:04.528687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:04.529004 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:04.529844 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:04.529882 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:04.534644 (+ 4762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:04.539871 (+ 5227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:04.539881 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:04.539897 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:04.540618 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:04.541296 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:04.541301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:04.541305 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:04.541618 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:04.542887 (+ 1269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:04.542909 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:04.543331 (+ 422us) client_negotiation.cc:769] Sending connection context
0926 20:49:04.543674 (+ 343us) client_negotiation.cc:240] Negotiation successful
0926 20:49:04.544301 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13440,"thread_start_us":13166,"threads_started":1}
I20260926 20:49:04.548518 18541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:04.497632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:35109 (local address 127.5.78.62:37875)
0926 20:49:04.516113 (+ 18481us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:04.516136 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:04.516208 (+ 72us) server_negotiation.cc:407] Connection header received
0926 20:49:04.518640 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:04.518662 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:04.519266 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:04.519729 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:04.522372 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:04.524391 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:04.527005 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:04.528073 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:04.528261 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:04.535103 (+ 6842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:04.535117 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:04.535143 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:04.535208 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:04.539597 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:04.540845 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:04.540856 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:04.540864 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:04.541111 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:04.541794 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:04.541801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:04.541805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:04.542571 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:04.542777 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:04.546556 (+ 3779us) server_negotiation.cc:299] Negotiation successful
0926 20:49:04.547105 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17878,"thread_start_us":17900,"threads_started":1}
I20260926 20:49:04.553510 18538 heartbeater.cc:347] Connected to a master server at 127.5.78.62:37875
I20260926 20:49:04.553854 18538 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:04.554868 18538 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
W20260926 20:49:04.559036 18538 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:37875 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:49:04.559329 18538 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:04.559666 18538 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:04.561436 18538 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:04.561918 18538 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
W20260926 20:49:04.563457 18538 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:49:04.953366 18540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:04.953893 18540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:04.954108 18540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:04.966434 18540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:04.966665 18540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:04.966893 18540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:49:04.983429 18540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:34865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=39277
--tserver_master_addrs=127.5.78.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:04.985237 18540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:04.987072 18540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:05.006145 18558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:05.011785 18559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:05.020277 18561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:05.020773 18540 server_base.cc:1034] running on GCE node
I20260926 20:49:05.021667 18540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:05.023422 18540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:05.024756 18540 hybrid_clock.cc:648] HybridClock initialized: now 1790455745024662 us; error 81 us; skew 500 ppm
I20260926 20:49:05.025219 18540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:05.030064 18540 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:49:05.032485 18540 webserver.cc:460] Webserver started at http://127.5.78.2:39277/ using document root <none> and password file <none>
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:05.034088 18540 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:05.034273 18540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:05.067627 18540 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260926 20:49:05.074103 18569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:05.076599 18540 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.002s
I20260926 20:49:05.076886 18540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "7633870685e64abe87558d507e8e4c33"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "55c5b669dcf2f7b7b76f2601fcc621d1"
server_key_iv: "5c343bd4e101a4df4b0a580e6f256b31"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:05.078740 18540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:05.097997 18540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:05.103462 18540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:05.103811 18540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:05.105154 18540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:05.107108 18540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:05.107239 18540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:05.107378 18540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:05.107458 18540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:05.170526 18540 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:34865
I20260926 20:49:05.170588 18682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:34865 every 8 connection(s)
I20260926 20:49:05.172631 18540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:49:05.180382 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18540
I20260926 20:49:05.181222 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.78.20:35107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:43675
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:37875
--webserver_port=44017
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:05.199352 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:05.176366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:37875 (local address 127.5.78.2:59091)
0926 20:49:05.177207 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:05.177254 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:05.179065 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:05.179911 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:05.179921 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:05.180386 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:05.181103 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:05.181123 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:05.182355 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:05.182360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:05.184152 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:05.184167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:05.185060 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:05.185069 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:05.185393 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:05.186162 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:05.186192 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:05.189088 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:05.193709 (+ 4621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:05.193722 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:05.193745 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:05.194580 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:05.195116 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:05.195121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:05.195125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:05.195426 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:05.196234 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:05.196246 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:05.196767 (+ 521us) client_negotiation.cc:769] Sending connection context
0926 20:49:05.197150 (+ 383us) client_negotiation.cc:240] Negotiation successful
0926 20:49:05.197760 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":329,"threads_started":1}
I20260926 20:49:05.200521 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:05.175686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:59091 (local address 127.5.78.62:37875)
0926 20:49:05.176163 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:05.176168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:05.177407 (+ 1239us) server_negotiation.cc:407] Connection header received
0926 20:49:05.179379 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:05.179385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:05.179480 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:05.179612 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:05.181327 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:05.182189 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:05.184320 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:05.184795 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:05.185001 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:05.189429 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:05.189437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:05.189443 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:05.189491 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:05.193435 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:05.194764 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:05.194770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:05.194774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:05.194951 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:05.195601 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:05.195606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:05.195610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:05.196004 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:05.196151 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:05.199839 (+ 3688us) server_negotiation.cc:299] Negotiation successful
0926 20:49:05.200122 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":258,"threads_started":1}
I20260926 20:49:05.202651 18683 heartbeater.cc:347] Connected to a master server at 127.5.78.62:37875
I20260926 20:49:05.203027 18683 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:05.204053 18683 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:05.206593 18341 ts_manager.cc:194] Registered new tserver with Master: 7633870685e64abe87558d507e8e4c33 (127.5.78.2:34865)
I20260926 20:49:05.208921 18341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:59091
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:05.563627 18538 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:05.564071 18538 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:05.565353 18341 ts_manager.cc:194] Registered new tserver with Master: 7146febe7ff14bc0969cf39fa17f13fa (127.5.78.1:34447)
I20260926 20:49:05.566501 18341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:35109
W20260926 20:49:05.683563 18687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:05.684084 18687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:05.684307 18687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:05.696386 18687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:05.696638 18687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:05.696859 18687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:49:05.713915 18687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.78.20:35107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:43675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=44017
--tserver_master_addrs=127.5.78.62:37875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:05.716192 18687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:05.718153 18687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:05.784056 18692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:05.780221 18693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:05.784452 18696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:05.785691 18687 server_base.cc:1034] running on GCE node
I20260926 20:49:05.786620 18687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:05.801244 18687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:05.804632 18687 hybrid_clock.cc:648] HybridClock initialized: now 1790455745804502 us; error 115 us; skew 500 ppm
I20260926 20:49:05.805155 18687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:05.809712 18687 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:49:05.830729 18687 webserver.cc:460] Webserver started at http://127.5.78.3:44017/ using document root <none> and password file <none>
I20260926 20:49:05.831928 18687 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:05.832195 18687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:05.881481 18687 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.004s
I20260926 20:49:05.895155 18704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:05.897066 18687 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.005s sys 0.000s
I20260926 20:49:05.897305 18687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "f3441e7d99884bde8e1696c91bce5232"
format_stamp: "Formatted at 2026-09-26 20:49:02 on dist-test-slave-z7j8"
server_key: "9b007c22c8f2b5ef8f699b75ae6347af"
server_key_iv: "b1b809ede7a72e4cf9a35ed7d7a92c70"
server_key_version: "kuduclusterkey@0"
I20260926 20:49:05.898500 18687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 26 20:49:05 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for HTTP/127.5.78.20@KRBTEST.COM
I20260926 20:49:05.935541 18687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:05.941386 18687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:05.941757 18687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:05.943056 18687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:05.944896 18687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:05.945029 18687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:05.945212 18687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:05.945308 18687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:49:06.132776 18687 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:43675
I20260926 20:49:06.132859 18816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:43675 every 8 connection(s)
I20260926 20:49:06.134781 18687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:49:06.136394 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18687
Sep 26 20:49:06 dist-test-slave-z7j8 krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:06.175701 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.137559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.5.78.62:37875)
0926 20:49:06.152566 (+ 15007us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.152577 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.152613 (+ 36us) server_negotiation.cc:407] Connection header received
0926 20:49:06.152722 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.152730 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.152872 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.153055 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:06.158015 (+ 4960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.159409 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.161482 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.162248 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.162394 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.166161 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:06.166168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:06.166173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:06.166235 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:06.171037 (+ 4802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.172149 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.172155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.172158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.172328 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.172879 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.172885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.172888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.173257 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:06.173530 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.175056 (+ 1526us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.175319 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":14813,"thread_start_us":14664,"threads_started":1}
I20260926 20:49:06.185937 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.145442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:55511 (local address 127.5.78.62:37875)
0926 20:49:06.159143 (+ 13701us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.159150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.159177 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:49:06.159235 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.159241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.159338 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.161850 (+ 2512us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:06.162149 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.163381 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.166172 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.166956 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.168450 (+ 1494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.177294 (+ 8844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:06.177301 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:06.177305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:06.177351 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:06.182308 (+ 4957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.183279 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.183284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.183287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.183437 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.184055 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.184060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.184062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.184388 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:06.184548 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.185409 (+ 861us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.185629 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13566,"thread_start_us":14018,"threads_started":1}
I20260926 20:49:06.187067 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.146267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:37875 (local address 127.5.78.3:55511)
0926 20:49:06.154100 (+ 7833us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.154169 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:06.156826 (+ 2657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:06.159995 (+ 3169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:06.160011 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:06.160697 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:06.161654 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.161684 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.163595 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.163603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.165948 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.165973 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.167310 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.167322 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.167931 (+ 609us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.168903 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:06.168944 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:06.176969 (+ 8025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:06.182564 (+ 5595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:06.182588 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:06.182604 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:06.183110 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:06.183631 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:06.183636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:06.183647 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:06.183893 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:06.184598 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:06.184605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:06.184966 (+ 361us) client_negotiation.cc:769] Sending connection context
0926 20:49:06.185297 (+ 331us) client_negotiation.cc:240] Negotiation successful
0926 20:49:06.185849 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7429,"thread_start_us":7224,"threads_started":1}
I20260926 20:49:06.188720 18817 heartbeater.cc:347] Connected to a master server at 127.5.78.62:37875
I20260926 20:49:06.189097 18817 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:06.189934 18817 heartbeater.cc:511] Master 127.5.78.62:37875 requested a full tablet report, sending...
I20260926 20:49:06.191948 18341 ts_manager.cc:194] Registered new tserver with Master: f3441e7d99884bde8e1696c91bce5232 (127.5.78.3:43675)
I20260926 20:49:06.192885 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:49:06.192994 18341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:55511
I20260926 20:49:06.210906 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.196415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.5.78.62:37875)
0926 20:49:06.196657 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.196667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.197141 (+ 474us) server_negotiation.cc:407] Connection header received
0926 20:49:06.197358 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.197364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.197448 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.197552 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:06.199307 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.200008 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.201577 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.202088 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.202237 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.205077 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:06.205082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:06.205086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:06.205141 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:06.207939 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.208764 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.208769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.208772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.208941 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.209361 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.209365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.209367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.209703 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:06.209891 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.210277 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.210529 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:49:06.214188 18683 heartbeater.cc:503] Master 127.5.78.62:37875 was elected leader, sending a full tablet report...
I20260926 20:49:06.216120 18340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:49:06.223884 18340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:49:06.259080 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.238719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.5.78.3:43675)
0926 20:49:06.249345 (+ 10626us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.249357 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.250120 (+ 763us) server_negotiation.cc:407] Connection header received
0926 20:49:06.250485 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.250493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.250764 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.250939 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:06.252311 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.253862 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.256374 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.257542 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.257710 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.257991 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.258468 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.258683 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10408,"thread_start_us":10227,"threads_started":1}
I20260926 20:49:06.259183 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.239877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.5.78.1:34447)
0926 20:49:06.248958 (+ 9081us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.248970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.250346 (+ 1376us) server_negotiation.cc:407] Connection header received
0926 20:49:06.250479 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.250486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.250765 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.250962 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:06.251985 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.253641 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.256431 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.258005 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.258165 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.258427 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.258663 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.258835 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8874,"thread_start_us":8694,"threads_started":1}
I20260926 20:49:06.259357 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.239257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56044)
0926 20:49:06.249975 (+ 10718us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.250008 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:06.250209 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:06.251202 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:06.251207 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:06.251257 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:06.251794 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.251806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.253784 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.253789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.256255 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.256267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.258269 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.258275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.258485 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.258514 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 20:49:06.258603 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:49:06.258801 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10545,"mutex_wait_us":60,"thread_start_us":10367,"threads_started":1}
I20260926 20:49:06.260285 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.238400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:43675 (local address 127.0.0.1:46642)
0926 20:49:06.249958 (+ 11558us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.250011 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:06.250253 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:06.251304 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:06.251308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:06.251336 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:06.252095 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.252111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.254040 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.254044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.256188 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.256207 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.257783 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.257790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.258136 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.258211 (+ 75us) client_negotiation.cc:769] Sending connection context
0926 20:49:06.259704 (+ 1493us) client_negotiation.cc:240] Negotiation successful
0926 20:49:06.259886 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11248,"mutex_wait_us":147,"thread_start_us":11013,"threads_started":1}
I20260926 20:49:06.261037 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.251722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:34865 (local address 127.0.0.1:39364)
0926 20:49:06.252356 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.252374 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:06.252494 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:06.253040 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:06.253044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:06.253068 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:06.253439 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.253448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.255421 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.255427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.258103 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.258118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.260128 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.260135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.260377 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.260420 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 20:49:06.260520 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 20:49:06.260647 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":29,"thread_start_us":526,"threads_started":1}
I20260926 20:49:06.261116 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.251064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.5.78.2:34865)
0926 20:49:06.251722 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.251732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.252484 (+ 752us) server_negotiation.cc:407] Connection header received
0926 20:49:06.252666 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.252672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.252866 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.253056 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:06.253617 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.255271 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.258296 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.259877 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.260039 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.260279 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.260593 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.260778 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":468,"threads_started":1}
I20260926 20:49:06.263787 18473 tablet_service.cc:1467] Processing CreateTablet for tablet 70ca9e1da50345cd93a968322e21d784 (DEFAULT_TABLE table=test-table [id=8898af069e8c42cebc2d95da8d04e332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:06.265071 18473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ca9e1da50345cd93a968322e21d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:06.264631 18618 tablet_service.cc:1467] Processing CreateTablet for tablet 70ca9e1da50345cd93a968322e21d784 (DEFAULT_TABLE table=test-table [id=8898af069e8c42cebc2d95da8d04e332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:06.264973 18752 tablet_service.cc:1467] Processing CreateTablet for tablet 70ca9e1da50345cd93a968322e21d784 (DEFAULT_TABLE table=test-table [id=8898af069e8c42cebc2d95da8d04e332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:06.266077 18618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ca9e1da50345cd93a968322e21d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:06.266211 18752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ca9e1da50345cd93a968322e21d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:06.278695 18845 tablet_bootstrap.cc:492] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: Bootstrap starting.
I20260926 20:49:06.282284 18845 tablet_bootstrap.cc:654] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:06.282354 18846 tablet_bootstrap.cc:492] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Bootstrap starting.
I20260926 20:49:06.284288 18847 tablet_bootstrap.cc:492] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Bootstrap starting.
I20260926 20:49:06.285682 18845 log.cc:824] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:06.286167 18846 tablet_bootstrap.cc:654] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:06.287338 18846 log.cc:824] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:06.287912 18847 tablet_bootstrap.cc:654] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:06.289079 18847 log.cc:824] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:06.289199 18845 tablet_bootstrap.cc:492] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: No bootstrap required, opened a new log
I20260926 20:49:06.289484 18845 ts_tablet_manager.cc:1397] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260926 20:49:06.290571 18846 tablet_bootstrap.cc:492] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: No bootstrap required, opened a new log
I20260926 20:49:06.290824 18846 ts_tablet_manager.cc:1397] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 20:49:06.291906 18847 tablet_bootstrap.cc:492] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: No bootstrap required, opened a new log
I20260926 20:49:06.292156 18847 ts_tablet_manager.cc:1397] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 20:49:06.299891 18847 raft_consensus.cc:348] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.300303 18847 raft_consensus.cc:374] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:06.300400 18847 raft_consensus.cc:729] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3441e7d99884bde8e1696c91bce5232, State: Initialized, Role: FOLLOWER
I20260926 20:49:06.301096 18847 consensus_queue.cc:260] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.301170 18845 raft_consensus.cc:348] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.301716 18845 raft_consensus.cc:374] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:06.301851 18845 raft_consensus.cc:729] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7146febe7ff14bc0969cf39fa17f13fa, State: Initialized, Role: FOLLOWER
I20260926 20:49:06.301633 18846 raft_consensus.cc:348] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.302114 18846 raft_consensus.cc:374] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:06.302233 18846 raft_consensus.cc:729] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7633870685e64abe87558d507e8e4c33, State: Initialized, Role: FOLLOWER
I20260926 20:49:06.302740 18817 heartbeater.cc:503] Master 127.5.78.62:37875 was elected leader, sending a full tablet report...
I20260926 20:49:06.302982 18846 consensus_queue.cc:260] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.303308 18847 ts_tablet_manager.cc:1428] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260926 20:49:06.304299 18845 consensus_queue.cc:260] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.304841 18846 ts_tablet_manager.cc:1428] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260926 20:49:06.311151 18538 heartbeater.cc:503] Master 127.5.78.62:37875 was elected leader, sending a full tablet report...
I20260926 20:49:06.311726 18845 ts_tablet_manager.cc:1428] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: Time spent starting tablet: real 0.022s user 0.014s sys 0.007s
I20260926 20:49:06.348199 18852 raft_consensus.cc:482] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:49:06.348547 18852 raft_consensus.cc:504] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.350306 18852 leader_election.cc:288] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3441e7d99884bde8e1696c91bce5232 (127.5.78.3:43675), 7146febe7ff14bc0969cf39fa17f13fa (127.5.78.1:34447)
I20260926 20:49:06.360152 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.350858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:47121 (local address 127.5.78.3:43675)
0926 20:49:06.351040 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.351047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.351777 (+ 730us) server_negotiation.cc:407] Connection header received
0926 20:49:06.352098 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.352104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.352182 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.352274 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:06.353197 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.354302 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.357568 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.358858 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.359019 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.359123 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.359737 (+ 614us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.359884 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:49:06.360148 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.350821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:39719 (local address 127.5.78.1:34447)
0926 20:49:06.351041 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.351047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.351778 (+ 731us) server_negotiation.cc:407] Connection header received
0926 20:49:06.352042 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.352048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.352148 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.352270 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:06.353150 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.354286 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.357574 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.358856 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.359009 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.359124 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.359737 (+ 613us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.359884 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:49:06.360477 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.350659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.5.78.2:39719)
0926 20:49:06.351637 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.351658 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:06.351823 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:06.352305 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:06.352310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:06.352357 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:06.352933 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.352946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.354494 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.354499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.357359 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.357377 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.359187 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.359195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.359463 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.359575 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 20:49:06.359672 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:49:06.359922 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":365,"spinlock_wait_cycles":17152,"thread_start_us":585,"threads_started":1}
I20260926 20:49:06.360910 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.350724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:43675 (local address 127.5.78.2:47121)
0926 20:49:06.351637 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.351658 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:06.351864 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:06.352305 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:06.352310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:06.352359 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:06.352843 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.352863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.354493 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.354499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.357359 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:06.357377 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.359199 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.359204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:06.359446 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.359560 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 20:49:06.359672 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 20:49:06.359922 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":329,"spinlock_wait_cycles":51968,"thread_start_us":499,"threads_started":1}
I20260926 20:49:06.361632 18493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ca9e1da50345cd93a968322e21d784" candidate_uuid: "7633870685e64abe87558d507e8e4c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7146febe7ff14bc0969cf39fa17f13fa" is_pre_election: true
I20260926 20:49:06.362109 18772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ca9e1da50345cd93a968322e21d784" candidate_uuid: "7633870685e64abe87558d507e8e4c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3441e7d99884bde8e1696c91bce5232" is_pre_election: true
I20260926 20:49:06.362294 18493 raft_consensus.cc:2454] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7633870685e64abe87558d507e8e4c33 in term 0.
I20260926 20:49:06.362715 18772 raft_consensus.cc:2454] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7633870685e64abe87558d507e8e4c33 in term 0.
I20260926 20:49:06.363281 18570 leader_election.cc:302] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7146febe7ff14bc0969cf39fa17f13fa, 7633870685e64abe87558d507e8e4c33; no voters:
I20260926 20:49:06.363755 18852 raft_consensus.cc:2781] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:49:06.363914 18852 raft_consensus.cc:482] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:49:06.364022 18852 raft_consensus.cc:3037] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:06.366168 18852 raft_consensus.cc:504] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.366816 18852 leader_election.cc:288] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [CANDIDATE]: Term 1 election: Requested vote from peers f3441e7d99884bde8e1696c91bce5232 (127.5.78.3:43675), 7146febe7ff14bc0969cf39fa17f13fa (127.5.78.1:34447)
I20260926 20:49:06.367416 18772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ca9e1da50345cd93a968322e21d784" candidate_uuid: "7633870685e64abe87558d507e8e4c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3441e7d99884bde8e1696c91bce5232"
I20260926 20:49:06.367429 18493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ca9e1da50345cd93a968322e21d784" candidate_uuid: "7633870685e64abe87558d507e8e4c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7146febe7ff14bc0969cf39fa17f13fa"
I20260926 20:49:06.367693 18493 raft_consensus.cc:3037] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:06.367666 18772 raft_consensus.cc:3037] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:06.370270 18493 raft_consensus.cc:2454] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7633870685e64abe87558d507e8e4c33 in term 1.
I20260926 20:49:06.370322 18772 raft_consensus.cc:2454] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7633870685e64abe87558d507e8e4c33 in term 1.
I20260926 20:49:06.370791 18570 leader_election.cc:302] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7146febe7ff14bc0969cf39fa17f13fa, 7633870685e64abe87558d507e8e4c33; no voters:
I20260926 20:49:06.371177 18852 raft_consensus.cc:2781] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:06.371619 18852 raft_consensus.cc:686] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 1 LEADER]: Becoming Leader. State: Replica: 7633870685e64abe87558d507e8e4c33, State: Running, Role: LEADER
I20260926 20:49:06.372108 18852 consensus_queue.cc:237] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } }
I20260926 20:49:06.377276 18340 catalog_manager.cc:5495] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7633870685e64abe87558d507e8e4c33 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "7633870685e64abe87558d507e8e4c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7633870685e64abe87558d507e8e4c33" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 } health_report { overall_health: UNKNOWN } } }
W20260926 20:49:06.403712 18823 tablet.cc:2369] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:06.412225 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.406794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.5.78.2:34865)
0926 20:49:06.406993 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.406998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.407016 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:49:06.407194 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.407197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.407283 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.407414 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:49:06.407796 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.408578 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.410032 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.410514 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.410621 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.411109 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:49:06.411645 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:49:06.411754 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.411842 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.411910 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:49:06.420969 18493 raft_consensus.cc:1261] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 1 FOLLOWER]: Refusing update from remote peer 7633870685e64abe87558d507e8e4c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:06.420970 18772 raft_consensus.cc:1261] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 1 FOLLOWER]: Refusing update from remote peer 7633870685e64abe87558d507e8e4c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:06.422397 18852 consensus_queue.cc:1035] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7146febe7ff14bc0969cf39fa17f13fa" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 34447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:06.422838 18856 consensus_queue.cc:1035] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3441e7d99884bde8e1696c91bce5232" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 43675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 20:49:06.425299 18684 tablet.cc:2369] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:06.436059 18859 mvcc.cc:205] Tried to move back new op lower bound from 7333706737329790976 to 7333706737145044992. Current Snapshot: MvccSnapshot[applied={T|T < 7333706737329790976}]
I20260926 20:49:06.457262 18340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:06.457581 18340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:06.461529 18340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c45cc4addc8d4a7d8aea9affde069bc3: Sending DeleteTablet for 3 replicas of tablet 70ca9e1da50345cd93a968322e21d784
I20260926 20:49:06.462931 18618 tablet_service.cc:1514] Processing DeleteTablet for tablet 70ca9e1da50345cd93a968322e21d784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:06 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:39364
I20260926 20:49:06.462944 18473 tablet_service.cc:1514] Processing DeleteTablet for tablet 70ca9e1da50345cd93a968322e21d784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:06 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:56044
I20260926 20:49:06.462942 18752 tablet_service.cc:1514] Processing DeleteTablet for tablet 70ca9e1da50345cd93a968322e21d784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:06 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:46642
I20260926 20:49:06.465391 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18378
I20260926 20:49:06.495342 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.471265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.5.78.1:34447)
0926 20:49:06.471528 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.471538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.471568 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:49:06.471675 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.471683 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.471793 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.471944 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:06.474262 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.475829 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.478365 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.479350 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.479504 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.483828 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:06.483850 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:06.483878 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:06.483941 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:06.491005 (+ 7064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.492369 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.492385 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.492391 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.492661 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.493231 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.493240 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.493246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.493909 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:06.494224 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.494612 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.494898 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 20:49:06.496644 18533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:06.501372 18870 tablet_replica.cc:330] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: stopping tablet replica
I20260926 20:49:06.501595 18869 tablet_replica.cc:330] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: stopping tablet replica
I20260926 20:49:06.501945 18870 raft_consensus.cc:2229] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:06.501983 18869 raft_consensus.cc:2229] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:49:06.502579 18870 raft_consensus.cc:2258] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:06.502733 18869 raft_consensus.cc:2258] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:06.505400 18869 ts_tablet_manager.cc:1905] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:06.512826 18870 ts_tablet_manager.cc:1905] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:06.518044 18869 ts_tablet_manager.cc:1918] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:06.518270 18869 log.cc:1196] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/70ca9e1da50345cd93a968322e21d784
I20260926 20:49:06.519343 18869 ts_tablet_manager.cc:1939] T 70ca9e1da50345cd93a968322e21d784 P 7633870685e64abe87558d507e8e4c33: Deleting consensus metadata
I20260926 20:49:06.521452 18328 catalog_manager.cc:4841] TS 7633870685e64abe87558d507e8e4c33 (127.5.78.2:34865): tablet 70ca9e1da50345cd93a968322e21d784 (table test-table [id=8898af069e8c42cebc2d95da8d04e332]) successfully deleted
I20260926 20:49:06.522607 18870 ts_tablet_manager.cc:1918] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:06.522789 18870 log.cc:1196] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/70ca9e1da50345cd93a968322e21d784
I20260926 20:49:06.523248 18870 ts_tablet_manager.cc:1939] T 70ca9e1da50345cd93a968322e21d784 P f3441e7d99884bde8e1696c91bce5232: Deleting consensus metadata
I20260926 20:49:06.524775 18328 catalog_manager.cc:4841] TS f3441e7d99884bde8e1696c91bce5232 (127.5.78.3:43675): tablet 70ca9e1da50345cd93a968322e21d784 (table test-table [id=8898af069e8c42cebc2d95da8d04e332]) successfully deleted
I20260926 20:49:06.652311 18871 tablet_replica.cc:330] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa: stopping tablet replica
I20260926 20:49:06.652643 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18378
I20260926 20:49:06.652880 18871 raft_consensus.cc:2229] T 70ca9e1da50345cd93a968322e21d784 P 7146febe7ff14bc0969cf39fa17f13fa [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 20:49:06.669719 18325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.78.1:34447 (error 108)
I20260926 20:49:06.669869 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18540
I20260926 20:49:06.671955 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.671491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56060)
0926 20:49:06.671727 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.671818 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260926 20:49:06.672237 18325 catalog_manager.cc:4584] TS 7146febe7ff14bc0969cf39fa17f13fa (127.5.78.1:34447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70ca9e1da50345cd93a968322e21d784: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
I20260926 20:49:06.687492 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.670862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.5.78.2:34865)
0926 20:49:06.671050 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.671059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.671082 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:49:06.671281 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.671286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.671391 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.671492 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:06.673369 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.674393 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.676388 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.676886 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.676994 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.680539 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:06.680556 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:06.680581 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:06.680629 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:06.684123 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.685146 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.685158 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.685166 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.685339 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.685861 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.685865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.685867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.686286 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:06.686628 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.686952 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.687162 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:49:06.688285 18678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:06.725945 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.725462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56064)
0926 20:49:06.725679 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.725786 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:49:06.727201 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.726789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56072)
0926 20:49:06.726986 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.727082 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 20:49:06.787858 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.787381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56078)
0926 20:49:06.787605 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.787691 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 20:49:06.788924 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.788564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56090)
0926 20:49:06.788713 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.788790 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 20:49:06.817950 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18540
I20260926 20:49:06.837406 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18687
I20260926 20:49:06.851536 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.838156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.5.78.3:43675)
0926 20:49:06.838383 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:06.838387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:06.838457 (+ 70us) server_negotiation.cc:407] Connection header received
0926 20:49:06.838619 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:06.838623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:06.838686 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:06.838781 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:06.840307 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.841049 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.842464 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:06.842870 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:06.842970 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:06.845616 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:06.845631 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:06.845647 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:06.845682 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:06.848626 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.849401 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.849410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.849412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.849551 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:06.849984 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:06.849987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:06.849989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:06.850342 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:06.850702 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:06.850961 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 20:49:06.851138 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260926 20:49:06.852177 18812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:06.869556 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.868958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56094)
0926 20:49:06.869220 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.869343 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 20:49:06.870819 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:06.870322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56096)
0926 20:49:06.870571 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:06.870678 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260926 20:49:06.943819 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18687
I20260926 20:49:06.961462 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18308
I20260926 20:49:06.962812 18371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:07.079756 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:07.079135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:34447 (local address 127.0.0.1:56108)
0926 20:49:07.079340 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:07.079430 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:34447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 20:49:07.080341 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18308
I20260926 20:49:07.101910 5432 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 20:49:07.437860 5432 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 20:49:07.438115 5432 mini_ranger.cc:66] Stopping Ranger...
I20260926 20:49:07.784548 5432 mini_ranger.cc:68] Stopped Ranger
2026-09-26 20:49:07.784 UTC [17360] LOG: received smart shutdown request
2026-09-26 20:49:07.788 UTC [17360] LOG: background worker "logical replication launcher" (PID 17369) exited with exit code 1
2026-09-26 20:49:07.788 UTC [17364] LOG: shutting down
2026-09-26 20:49:07.999 UTC [17360] LOG: database system is shut down
2026-09-26T20:49:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55296 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:49:08 dist-test-slave-z7j8 krb5kdc[18885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:49:08 dist-test-slave-z7j8 krb5kdc[18885](info): set up 2 sockets
Sep 26 20:49:08 dist-test-slave-z7j8 krb5kdc[18885](info): commencing operation
krb5kdc: starting...
W20260926 20:49:09.235010 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:49:09 dist-test-slave-z7j8 krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:49:09.478590 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42703
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42703,127.5.78.61:33101,127.5.78.60:45121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:09.818051 18898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:09.818363 18898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:09.818532 18898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:09.825845 18898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:09.825964 18898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:09.826010 18898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:09.826061 18898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:09.826099 18898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:09.836138 18898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42703,127.5.78.61:33101,127.5.78.60:45121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:09.837431 18898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:09.838857 18898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:09.851128 18903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:09.851063 18904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:09.851094 18906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:09.851704 18898 server_base.cc:1034] running on GCE node
I20260926 20:49:09.852628 18898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:09.854210 18898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:09.855469 18898 hybrid_clock.cc:648] HybridClock initialized: now 1790455749855387 us; error 75 us; skew 500 ppm
I20260926 20:49:09.855901 18898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:09 dist-test-slave-z7j8 krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:09.864857 18898 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:09.866736 18898 webserver.cc:460] Webserver started at http://127.5.78.62:33325/ using document root <none> and password file <none>
I20260926 20:49:09.867528 18898 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:09.867703 18898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:09.868037 18898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:09.871572 18898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "130195e1c55a467ea2a8b2cb42692506"
format_stamp: "Formatted at 2026-09-26 20:49:09 on dist-test-slave-z7j8"
server_key: "8c19b4e22e4039ba9e0a601270784df1"
server_key_iv: "d1fa2d59f9e66b6def5cbd44b4f21fc3"
server_key_version: "encryptionkey@0"
I20260926 20:49:09.872502 18898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "130195e1c55a467ea2a8b2cb42692506"
format_stamp: "Formatted at 2026-09-26 20:49:09 on dist-test-slave-z7j8"
server_key: "8c19b4e22e4039ba9e0a601270784df1"
server_key_iv: "d1fa2d59f9e66b6def5cbd44b4f21fc3"
server_key_version: "encryptionkey@0"
I20260926 20:49:09.878775 18898 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 20:49:09.882704 18913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:09.884223 18898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:49:09.884454 18898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "130195e1c55a467ea2a8b2cb42692506"
format_stamp: "Formatted at 2026-09-26 20:49:09 on dist-test-slave-z7j8"
server_key: "8c19b4e22e4039ba9e0a601270784df1"
server_key_iv: "d1fa2d59f9e66b6def5cbd44b4f21fc3"
server_key_version: "encryptionkey@0"
I20260926 20:49:09.884680 18898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:09.917989 18898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:09.922830 18898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:09.923139 18898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:09.943276 18898 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:42703
I20260926 20:49:09.943320 18964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:42703 every 8 connection(s)
I20260926 20:49:09.944780 18898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:09.948902 18965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:09.951936 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18898
I20260926 20:49:09.952194 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:49:09.952836 5432 external_mini_cluster.cc:1428] Setting key a6339ec8046a1390b4204a385a5267db
I20260926 20:49:09.956120 18965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:49:09 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.61@KRBTEST.COM" created.
I20260926 20:49:09.979986 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:09.960733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.5.78.62:42703)
0926 20:49:09.961480 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:09.961489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:09.961557 (+ 68us) server_negotiation.cc:407] Connection header received
0926 20:49:09.963124 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:09.963130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:09.963408 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:09.963641 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:09.964982 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:09.966227 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:09.967711 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:09.968279 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:09.968506 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:09.972216 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:09.972224 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:09.972237 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:09.972267 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:09.975143 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:09.976049 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:09.976061 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:09.976064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:09.976205 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:09.976607 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:09.976616 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:09.976619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:09.977237 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:09.977457 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:09.978016 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 20:49:09.978447 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":534,"thread_start_us":372,"threads_started":1}
I20260926 20:49:09.980018 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:09.959540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42703 (local address 127.0.0.1:54570)
0926 20:49:09.960376 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:09.960420 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:09.962688 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:09.963659 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:09.963666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:09.964430 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:09.964773 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:09.964792 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:09.966393 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:09.966428 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:09.967562 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:09.967574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:09.968528 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:09.968535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:09.968711 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:09.969462 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:09.969486 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:09.971972 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:09.975280 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:09.975293 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:09.975306 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:09.975920 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:09.976326 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:09.976331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:09.976335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:09.976493 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:09.977372 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:09.977385 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:09.977633 (+ 248us) client_negotiation.cc:769] Sending connection context
0926 20:49:09.977885 (+ 252us) client_negotiation.cc:240] Negotiation successful
0926 20:49:09.978564 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":188,"thread_start_us":342,"threads_started":1}
I20260926 20:49:09.983908 18965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:49:09.985033 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:09.984628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38310)
0926 20:49:09.984798 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:09.984913 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:49:09.985350 18915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
I20260926 20:49:09.986994 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:09.986639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38322)
0926 20:49:09.986782 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:09.986864 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 20:49:09.987217 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:33101: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab.
Entry for principal kudu/127.5.78.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab.
I20260926 20:49:10.004245 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:33101
--webserver_interface=127.5.78.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab
--principal=kudu/127.5.78.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42703,127.5.78.61:33101,127.5.78.60:45121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:49:10.036537 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } attempt: 1
I20260926 20:49:10.037815 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.037298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38326)
0926 20:49:10.037548 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.037675 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260926 20:49:10.039155 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.038673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38338)
0926 20:49:10.038849 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.038941 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 20:49:10.039359 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:33101: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
I20260926 20:49:10.107650 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } attempt: 2
I20260926 20:49:10.108884 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.108372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38342)
0926 20:49:10.108602 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.108713 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:49:10.109974 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.109529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38356)
0926 20:49:10.109681 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.109784 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:49:10.110232 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:33101: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
I20260926 20:49:10.201558 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } attempt: 3
I20260926 20:49:10.202940 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.202399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38360)
0926 20:49:10.202688 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.202791 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 20:49:10.204135 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.203707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38368)
0926 20:49:10.203873 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.203942 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:49:10.204334 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:33101: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
W20260926 20:49:10.341681 18972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:10.342048 18972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:10.342217 18972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 20:49:10.347581 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } attempt: 4
I20260926 20:49:10.348783 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.348219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38384)
0926 20:49:10.348569 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.348655 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260926 20:49:10.349946 18972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260926 20:49:10.350018 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.349579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38386)
0926 20:49:10.349763 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.349884 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 20:49:10.350085 18972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:10.350185 18972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:10.350274 18972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:10.350339 18972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:49:10.350373 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:33101: Network error: Client connection negotiation failed: client connection to 127.5.78.61:33101: connect: Connection refused (error 111)
I20260926 20:49:10.360522 18972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42703,127.5.78.61:33101,127.5.78.60:45121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab
--principal=kudu/127.5.78.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:33101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.78.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:10.361860 18972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:10.363255 18972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:10.375998 18977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:10.375949 18978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:10.376020 18980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:10.376842 18972 server_base.cc:1034] running on GCE node
I20260926 20:49:10.377691 18972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:10.379419 18972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:10.380668 18972 hybrid_clock.cc:648] HybridClock initialized: now 1790455750380578 us; error 113 us; skew 500 ppm
I20260926 20:49:10.381078 18972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:10.384649 18972 init.cc:377] Logged in from keytab as kudu/127.5.78.61@KRBTEST.COM (short username kudu)
I20260926 20:49:10.386453 18972 webserver.cc:460] Webserver started at http://127.5.78.61:35563/ using document root <none> and password file <none>
I20260926 20:49:10.387208 18972 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:10.387315 18972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:10.387586 18972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:10.390921 18972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data/instance:
uuid: "4932e27c4d2c4d0cb9ff548b255edfa0"
format_stamp: "Formatted at 2026-09-26 20:49:10 on dist-test-slave-z7j8"
server_key: "300a0b63b78306708c3be9785f522c7d"
server_key_iv: "72ccb2ff3bb4f0a05d836eddf8ea7796"
server_key_version: "encryptionkey@0"
I20260926 20:49:10.391747 18972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal/instance:
uuid: "4932e27c4d2c4d0cb9ff548b255edfa0"
format_stamp: "Formatted at 2026-09-26 20:49:10 on dist-test-slave-z7j8"
server_key: "300a0b63b78306708c3be9785f522c7d"
server_key_iv: "72ccb2ff3bb4f0a05d836eddf8ea7796"
server_key_version: "encryptionkey@0"
I20260926 20:49:10.397465 18972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:49:10.401405 18987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:10.403275 18972 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:10.403455 18972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal
uuid: "4932e27c4d2c4d0cb9ff548b255edfa0"
format_stamp: "Formatted at 2026-09-26 20:49:10 on dist-test-slave-z7j8"
server_key: "300a0b63b78306708c3be9785f522c7d"
server_key_iv: "72ccb2ff3bb4f0a05d836eddf8ea7796"
server_key_version: "encryptionkey@0"
I20260926 20:49:10.403641 18972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:10.418016 18972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:10.422861 18972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:10.423154 18972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:10.442751 18972 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.61:33101
I20260926 20:49:10.442796 19038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.61:33101 every 8 connection(s)
I20260926 20:49:10.444284 18972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
I20260926 20:49:10.446926 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 18972
I20260926 20:49:10.447149 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-1/wal/instance
I20260926 20:49:10.447634 5432 external_mini_cluster.cc:1428] Setting key 1a2021499da92c5aa611c35275780657
I20260926 20:49:10.448657 19039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:10.455696 19039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.78.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.60@KRBTEST.COM" created.
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:10.481918 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.458582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54584 (local address 127.5.78.62:42703)
0926 20:49:10.458819 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:10.458826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:10.460500 (+ 1674us) server_negotiation.cc:407] Connection header received
0926 20:49:10.463828 (+ 3328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:10.463833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:10.463914 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:10.464014 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:10.466503 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.467223 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.469392 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.470032 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.470151 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.474230 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:10.474240 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:10.474247 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:10.474308 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:10.478220 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.479381 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.479387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.479391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.479537 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.480091 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.480097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.480100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.480407 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:10.480585 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:10.481339 (+ 754us) server_negotiation.cc:299] Negotiation successful
0926 20:49:10.481583 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:49:10.483107 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.459185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42703 (local address 127.0.0.1:54584)
0926 20:49:10.460263 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.460321 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:10.463386 (+ 3065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:10.464355 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:10.464371 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:10.465274 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:10.466195 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.466224 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.467427 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.467432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.469198 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.469212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.470274 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.470280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.470563 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.471416 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:10.471443 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:10.473933 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:10.478457 (+ 4524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.478469 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.478490 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.479225 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.479670 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.479675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.479679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.479960 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.480544 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:10.480552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:10.480956 (+ 404us) client_negotiation.cc:769] Sending connection context
0926 20:49:10.481232 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 20:49:10.481783 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":489,"threads_started":1}
I20260926 20:49:10.485010 19039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab.
Entry for principal kudu/127.5.78.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab.
I20260926 20:49:10.502625 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:45121
--webserver_interface=127.5.78.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab
--principal=kudu/127.5.78.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42703,127.5.78.61:33101,127.5.78.60:45121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:49:10.503407 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.485881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38400)
0926 20:49:10.486089 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.486110 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:10.486234 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:10.487507 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:10.487510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:10.487917 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:10.488089 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.488098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.490461 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.490474 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.491360 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.491375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.492806 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.492813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.492971 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.493699 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:10.493720 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:10.496745 (+ 3025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:10.500535 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.500541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.500545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.500990 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.501543 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.501547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.501549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.501706 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.502364 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:10.502368 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:10.502557 (+ 189us) client_negotiation.cc:769] Sending connection context
0926 20:49:10.502701 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 20:49:10.503023 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 20:49:10.503564 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.486035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.5.78.61:33101)
0926 20:49:10.486860 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:10.486870 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:10.486916 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:49:10.486983 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:10.486994 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:10.487263 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:10.487468 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:10.488264 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.490189 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.491545 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.492581 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.492759 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.497040 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:10.497062 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:10.497077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:10.497113 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:10.500322 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.501180 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.501192 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.501198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.501407 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.501843 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.501851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.501855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.502228 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:10.502503 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:10.502868 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 20:49:10.503154 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":622,"thread_start_us":468,"threads_started":1}
I20260926 20:49:10.505895 19039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:49:10.507207 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.506695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38540)
0926 20:49:10.506908 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.507050 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 20:49:10.507577 18989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.509570 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.509018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38554)
0926 20:49:10.509236 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.509388 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 20:49:10.509852 19039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.559289 19039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 1
I20260926 20:49:10.560695 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.560133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38566)
0926 20:49:10.560391 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.560516 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260926 20:49:10.561898 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.561502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38574)
0926 20:49:10.561685 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.561774 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 20:49:10.562232 19039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.630502 19039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 2
I20260926 20:49:10.631753 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.631271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38590)
0926 20:49:10.631489 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.631596 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 20:49:10.632846 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.632528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38594)
0926 20:49:10.632668 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.632736 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 20:49:10.633112 19039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.649729 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } attempt: 5
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:49:10.663834 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.650855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.5.78.61:33101)
0926 20:49:10.651055 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:10.651060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:10.651083 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:49:10.651189 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:10.651200 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:10.651260 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:10.651376 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:10.652225 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.652942 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.654166 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.654720 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.654874 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.658159 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:10.658164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:10.658167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:10.658200 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:10.661130 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.661853 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.661861 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.661870 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.661992 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.662381 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.662388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.662391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.662725 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:10.662904 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:10.663270 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 20:49:10.663505 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:49:10.663836 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.650615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38414)
0926 20:49:10.650856 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.650879 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:10.651047 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:10.651409 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:10.651414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:10.651862 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:10.652053 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.652062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.653122 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.653130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.653988 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.654000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.654910 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.654917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.655108 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.655849 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:10.655868 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:10.657935 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:10.661299 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.661306 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.661313 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.661702 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.662111 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.662116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.662120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.662271 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.662851 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:10.662856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:10.663013 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 20:49:10.663165 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 20:49:10.663464 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 20:49:10.665398 18965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:49:10.666469 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.666023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38604)
0926 20:49:10.666187 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.666269 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 20:49:10.667393 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.667058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38612)
0926 20:49:10.667224 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.667288 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 20:49:10.667610 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.718914 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 1
I20260926 20:49:10.720330 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.719602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38628)
0926 20:49:10.719923 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.720071 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 20:49:10.721612 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.721138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38636)
0926 20:49:10.721327 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.721439 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:49:10.721882 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.724324 19039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 3
I20260926 20:49:10.725387 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.724954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38638)
0926 20:49:10.725145 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.725225 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 20:49:10.726467 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.726003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38648)
0926 20:49:10.726232 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.726304 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260926 20:49:10.726675 19039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.790131 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 2
I20260926 20:49:10.791399 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.790828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38658)
0926 20:49:10.791072 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.791190 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 20:49:10.792709 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.792271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38660)
0926 20:49:10.792443 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.792549 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:49:10.793025 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
W20260926 20:49:10.850798 19046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:10.851119 19046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:10.851277 19046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:10.859001 19046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:10.859131 19046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:10.859203 19046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:10.859292 19046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:10.859345 19046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:10.869890 19046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42703,127.5.78.61:33101,127.5.78.60:45121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab
--principal=kudu/127.5.78.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:45121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.78.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:10.869961 19039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 4
I20260926 20:49:10.871272 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.870714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38674)
0926 20:49:10.870962 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.871075 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 20:49:10.871527 19046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:10.872299 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.871944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38680)
0926 20:49:10.872096 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.872186 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 20:49:10.872501 19039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.873070 19046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:10.885840 19052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:10.886062 19051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:10.885810 19054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:10.886957 19046 server_base.cc:1034] running on GCE node
I20260926 20:49:10.888010 19046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:10.889752 19046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:10.890973 19046 hybrid_clock.cc:648] HybridClock initialized: now 1790455750890904 us; error 66 us; skew 500 ppm
I20260926 20:49:10.891387 19046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:10.894801 19046 init.cc:377] Logged in from keytab as kudu/127.5.78.60@KRBTEST.COM (short username kudu)
I20260926 20:49:10.896482 19046 webserver.cc:460] Webserver started at http://127.5.78.60:44565/ using document root <none> and password file <none>
I20260926 20:49:10.897243 19046 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:10.897401 19046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:10.897738 19046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:10.899288 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 3
I20260926 20:49:10.900488 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.899956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38682)
0926 20:49:10.900222 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.900306 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 20:49:10.901612 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.901231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38686)
0926 20:49:10.901420 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.901498 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:49:10.901917 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:45121: Network error: Client connection negotiation failed: client connection to 127.5.78.60:45121: connect: Connection refused (error 111)
I20260926 20:49:10.901556 19046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data/instance:
uuid: "16598372f92d463780b769327eb8c8ae"
format_stamp: "Formatted at 2026-09-26 20:49:10 on dist-test-slave-z7j8"
server_key: "5a4cd6f9a992e9930ffe193e95001e74"
server_key_iv: "48ecc03d2993d942745158d4aa8804d2"
server_key_version: "encryptionkey@0"
I20260926 20:49:10.902710 19046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal/instance:
uuid: "16598372f92d463780b769327eb8c8ae"
format_stamp: "Formatted at 2026-09-26 20:49:10 on dist-test-slave-z7j8"
server_key: "5a4cd6f9a992e9930ffe193e95001e74"
server_key_iv: "48ecc03d2993d942745158d4aa8804d2"
server_key_version: "encryptionkey@0"
I20260926 20:49:10.908733 19046 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 20:49:10.912766 19061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:10.914595 19046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:10.914822 19046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal
uuid: "16598372f92d463780b769327eb8c8ae"
format_stamp: "Formatted at 2026-09-26 20:49:10 on dist-test-slave-z7j8"
server_key: "5a4cd6f9a992e9930ffe193e95001e74"
server_key_iv: "48ecc03d2993d942745158d4aa8804d2"
server_key_version: "encryptionkey@0"
I20260926 20:49:10.915057 19046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:10.935072 19046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:10.939546 19046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:10.939872 19046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:10.960171 19046 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.60:45121
I20260926 20:49:10.960211 19112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.60:45121 every 8 connection(s)
I20260926 20:49:10.961615 19046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
I20260926 20:49:10.965685 19113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:10.967316 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19046
I20260926 20:49:10.967514 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455541382686-5432-0/minicluster-data/master-2/wal/instance
I20260926 20:49:10.968065 5432 external_mini_cluster.cc:1428] Setting key 7066fcd383b8c3b925d43314bf2a345e
I20260926 20:49:10.976651 19113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
Sep 26 20:49:10 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:10.996452 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.970282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.5.78.60:45121)
0926 20:49:10.971470 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:10.971491 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:10.971545 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:49:10.973444 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:10.973474 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:10.974008 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:10.974520 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:10.976788 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.978543 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.980558 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.981640 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.981811 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.986764 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:10.986773 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:10.986788 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:10.986837 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:10.991095 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.992048 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.992060 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.992074 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.992322 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.992842 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.992851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.992857 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.993717 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:10.994045 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:10.994518 (+ 473us) server_negotiation.cc:299] Negotiation successful
0926 20:49:10.995010 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":590,"threads_started":1}
I20260926 20:49:10.997643 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.980002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42703 (local address 127.0.0.1:54586)
0926 20:49:10.980919 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:10.980976 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:10.981213 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:10.981632 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:10.981640 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:10.982640 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:10.983078 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.983097 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.984343 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.984350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.985778 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:10.985795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.986765 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.986773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:10.986994 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.987766 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:10.987795 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:10.990909 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:10.994826 (+ 3917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.994841 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.994851 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.995364 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.995826 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:10.995829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:10.995831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:10.995968 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:10.996626 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:10.996637 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:10.996871 (+ 234us) client_negotiation.cc:769] Sending connection context
0926 20:49:10.997038 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 20:49:10.997336 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":459,"threads_started":1}
I20260926 20:49:10.997802 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:10.980115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.5.78.62:42703)
0926 20:49:10.980422 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:10.980431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:10.981105 (+ 674us) server_negotiation.cc:407] Connection header received
0926 20:49:10.981357 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:10.981364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:10.981445 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:10.981559 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:10.983390 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.984138 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.985962 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:10.986490 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:10.986643 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:10.991187 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:10.991194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:10.991197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:10.991238 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:10.994651 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.995512 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.995517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.995521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.995706 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:10.996085 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:10.996090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:10.996093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:10.996454 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:10.996629 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:10.997204 (+ 575us) server_negotiation.cc:299] Negotiation successful
0926 20:49:10.997451 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 20:49:10.999337 19113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:49:11 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
Sep 26 20:49:11 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:11.015031 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.000294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:33101 (local address 127.0.0.1:38430)
0926 20:49:11.000544 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:11.000565 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:11.000708 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:11.001012 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:11.001018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:11.001467 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:11.001639 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.001649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.003023 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.003029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.004243 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.004257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.005297 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.005301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.005499 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.006157 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:11.006185 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:11.008711 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:11.012422 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.012428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.012432 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.012869 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.013303 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.013309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.013312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.013476 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.014007 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:11.014012 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:11.014181 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 20:49:11.014331 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 20:49:11.014658 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260926 20:49:11.015058 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.000510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.5.78.61:33101)
0926 20:49:11.000729 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.000735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.000755 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:49:11.000813 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.000817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.000873 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.000977 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.001818 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.002859 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.004415 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.005106 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.005264 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.008941 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.008948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.008953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.009000 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.012242 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.013009 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.013015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.013019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.013180 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.013591 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.013598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.013601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.013882 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.014058 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.014444 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.014674 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:49:11.016471 19113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:49:11.017859 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.000278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.5.78.62:42703)
0926 20:49:11.000462 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.000470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.000489 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:49:11.000772 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.000780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.000876 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.000997 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.002642 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.003698 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.005480 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.006138 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.006284 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.011368 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.011374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.011377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.011418 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.014366 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.015294 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.015303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.015308 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.015495 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.015965 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.015970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.015973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.016359 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.016604 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.017060 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.017310 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 20:49:11 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
Sep 26 20:49:11 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:49:11.032123 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.017272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38700)
0926 20:49:11.017563 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:11.017581 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:11.017678 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:11.018026 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:11.018031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:11.018583 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:11.018776 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.018791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.020169 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.020177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.021617 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.021644 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.022590 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.022595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.022773 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.023359 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:11.023384 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:11.026016 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:11.029194 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.029202 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.029207 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.029645 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.030036 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.030044 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.030049 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.030207 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.030879 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:11.030883 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:11.031129 (+ 246us) client_negotiation.cc:769] Sending connection context
0926 20:49:11.031319 (+ 190us) client_negotiation.cc:240] Negotiation successful
0926 20:49:11.031661 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260926 20:49:11.032407 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.017435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.5.78.60:45121)
0926 20:49:11.017590 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.017595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.017615 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:49:11.017824 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.017828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.017886 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.018013 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.019033 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.020023 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.021767 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.022455 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.022574 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.026225 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.026230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.026233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.026262 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.028935 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.029774 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.029782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.029787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.029917 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.030321 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.030329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.030333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.030718 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.030906 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.031440 (+ 534us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.031843 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":179,"server-negotiator.queue_time_us":61}
I20260926 20:49:11.036410 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.019644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.5.78.61:33101)
0926 20:49:11.019904 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.019910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.019926 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:49:11.019975 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.019982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.020051 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.020152 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.021771 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.022842 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.024788 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.025460 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.025606 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.030227 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.030232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.030235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.030268 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.033623 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.034396 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.034424 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.034431 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.034576 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.034982 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.034987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.034990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.035274 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.035552 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.035845 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.036085 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260926 20:49:11.037494 19113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: Bootstrap starting.
I20260926 20:49:11.041642 19113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:11.041689 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:49:11.042851 19113 log.cc:824] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:11.045771 19113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: No bootstrap required, opened a new log
I20260926 20:49:11.059906 19113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.060425 19113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:11.060587 19113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16598372f92d463780b769327eb8c8ae, State: Initialized, Role: FOLLOWER
I20260926 20:49:11.061529 19113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.067509 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.047089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.5.78.61:33101)
0926 20:49:11.047289 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.047298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.048405 (+ 1107us) server_negotiation.cc:407] Connection header received
0926 20:49:11.048685 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.048693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.048781 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.048852 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.051130 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.052191 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.054378 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.055051 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.055146 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.059405 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.059412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.059419 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.059467 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.063391 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.064651 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.064659 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.064663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.064848 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.065370 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.065379 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.065384 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.065843 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.066097 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.066873 (+ 776us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.067134 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:49:11.067605 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.047440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.5.78.62:42703)
0926 20:49:11.047632 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.047638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.047899 (+ 261us) server_negotiation.cc:407] Connection header received
0926 20:49:11.048163 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.048171 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.048265 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.048349 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.050788 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.051941 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.054240 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.054949 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.055045 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.059667 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.059674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.059680 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.059723 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.063851 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.064932 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.064938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.064941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.065126 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.065692 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.065699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.065704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.066111 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.066361 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.066973 (+ 612us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.067245 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 20:49:11.070624 19126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.071277 19126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.072086 19113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:11.074069 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.048808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38710 (local address 127.5.78.60:45121)
0926 20:49:11.050063 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.050073 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.056444 (+ 6371us) server_negotiation.cc:407] Connection header received
0926 20:49:11.056773 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.056782 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.056873 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.056960 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.059321 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.060463 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.062609 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.063350 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.063464 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.067071 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.067078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.067083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.067127 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.070817 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.071737 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.071743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.071746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.071942 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.072406 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.072411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.072413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.072732 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.072964 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.073373 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.073659 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114}
I20260926 20:49:11.079142 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 4
W20260926 20:49:11.087550 19137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:49:11.087845 19137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 20:49:11 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455749, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
I20260926 20:49:11.080441 19046 master_runner.cc:186] Error getting master registration for 127.5.78.62:42703: OK, instance_id { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" instance_seqno: 1790455749937406 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:49:11.093556 19046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:49:11.097796 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.080437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38712)
0926 20:49:11.080642 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:11.080662 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:11.080786 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:11.081148 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:11.081153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:11.081675 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:11.081896 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.081907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.083505 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.083511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.085028 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.085046 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.086396 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.086430 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.086667 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.087379 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:11.087409 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:11.090341 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:11.094553 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.094559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.094563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.095111 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.095706 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.095712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.095716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.095943 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.096831 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:11.096837 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:11.097063 (+ 226us) client_negotiation.cc:769] Sending connection context
0926 20:49:11.097234 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 20:49:11.097505 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 20:49:11.098122 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.080463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.5.78.60:45121)
0926 20:49:11.080634 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.080640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.080745 (+ 105us) server_negotiation.cc:407] Connection header received
0926 20:49:11.080914 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.080919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.080994 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.081132 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.082073 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.083327 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.085225 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.086007 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.086158 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.090609 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.090617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.090622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.090665 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.094331 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.095303 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.095310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.095313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.095503 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.096147 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.096153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.096158 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.096608 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.096763 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.097366 (+ 603us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.097611 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:49:11.104379 18965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: Bootstrap starting.
I20260926 20:49:11.108868 18965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:11.110351 18965 log.cc:824] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:11.115144 18965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: No bootstrap required, opened a new log
I20260926 20:49:11.126685 18965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.127131 18965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:11.127269 18965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130195e1c55a467ea2a8b2cb42692506, State: Initialized, Role: FOLLOWER
I20260926 20:49:11.127987 18965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.128988 19140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.129360 19140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.130532 18965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:49:11.142885 19151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:49:11.143208 19151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:49:11.142112 18898 master_runner.cc:186] Error getting master registration for 127.5.78.61:33101: OK, instance_id { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" instance_seqno: 1790455750437072 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:49:11.151885 18898 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:49:11.171782 19039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } attempt: 5
W20260926 20:49:11.176535 18919 tablet.cc:2369] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 20:49:11 dist-test-slave-z7j8 krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455750, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
I20260926 20:49:11.189117 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.172571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:45121 (local address 127.0.0.1:38722)
0926 20:49:11.172881 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:11.172904 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:11.173051 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:11.173446 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:11.173451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:11.173891 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:11.174060 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.174069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.175248 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.175254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.176545 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:11.176562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.177796 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.177803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:11.178026 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.180016 (+ 1990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:11.180044 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:11.182398 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:11.185983 (+ 3585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.185989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.185993 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.186559 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.187154 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:11.187161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:11.187175 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:11.187357 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:11.187917 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:11.187922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:11.188131 (+ 209us) client_negotiation.cc:769] Sending connection context
0926 20:49:11.188313 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 20:49:11.188742 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260926 20:49:11.189100 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:11.172665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.5.78.60:45121)
0926 20:49:11.172910 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:11.172915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:11.172992 (+ 77us) server_negotiation.cc:407] Connection header received
0926 20:49:11.173213 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:11.173219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:11.173296 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:11.173407 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:11.174223 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.175075 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.176718 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:11.177394 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:11.177522 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:11.182728 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:11.182734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:11.182738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:11.182774 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:11.185796 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.186824 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.186847 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.186852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.187013 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:11.187475 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:11.187483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:11.187492 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:11.187789 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:11.187977 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:11.188416 (+ 439us) server_negotiation.cc:299] Negotiation successful
0926 20:49:11.188668 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
W20260926 20:49:11.192150 19067 tablet.cc:2369] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:11.195312 19039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: Bootstrap starting.
I20260926 20:49:11.196063 19126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:49:11.196393 19126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.198144 19126 leader_election.cc:288] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 130195e1c55a467ea2a8b2cb42692506 (127.5.78.62:42703), 4932e27c4d2c4d0cb9ff548b255edfa0 (127.5.78.61:33101)
I20260926 20:49:11.199045 18940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "16598372f92d463780b769327eb8c8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130195e1c55a467ea2a8b2cb42692506" is_pre_election: true
I20260926 20:49:11.199707 18940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16598372f92d463780b769327eb8c8ae in term 0.
I20260926 20:49:11.200449 19062 leader_election.cc:302] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130195e1c55a467ea2a8b2cb42692506, 16598372f92d463780b769327eb8c8ae; no voters:
I20260926 20:49:11.201179 19126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:49:11.201346 19126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:49:11.201452 19126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:11.202733 19039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:11.204952 19039 log.cc:824] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:11.205147 19126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.206956 18940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "16598372f92d463780b769327eb8c8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130195e1c55a467ea2a8b2cb42692506"
I20260926 20:49:11.207280 18940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:11.212813 18940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16598372f92d463780b769327eb8c8ae in term 1.
I20260926 20:49:11.213658 19062 leader_election.cc:302] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130195e1c55a467ea2a8b2cb42692506, 16598372f92d463780b769327eb8c8ae; no voters:
I20260926 20:49:11.215557 19126 leader_election.cc:288] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [CANDIDATE]: Term 1 election: Requested vote from peers 130195e1c55a467ea2a8b2cb42692506 (127.5.78.62:42703), 4932e27c4d2c4d0cb9ff548b255edfa0 (127.5.78.61:33101)
I20260926 20:49:11.216468 19154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:11.219424 19039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: No bootstrap required, opened a new log
I20260926 20:49:11.219167 19154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [term 1 LEADER]: Becoming Leader. State: Replica: 16598372f92d463780b769327eb8c8ae, State: Running, Role: LEADER
I20260926 20:49:11.220986 19154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.231595 19039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.232124 19039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:11.232280 19039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e27c4d2c4d0cb9ff548b255edfa0, State: Initialized, Role: FOLLOWER
I20260926 20:49:11.233382 19039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } }
I20260926 20:49:11.240808 19156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.241367 19156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.243640 19039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:11.244269 19126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16598372f92d463780b769327eb8c8ae. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.244621 19126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:11.244992 19014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "16598372f92d463780b769327eb8c8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" is_pre_election: true
I20260926 20:49:11.245674 19014 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16598372f92d463780b769327eb8c8ae in term 0.
I20260926 20:49:11.246193 19013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "16598372f92d463780b769327eb8c8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0"
I20260926 20:49:11.246492 19013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:11.251757 19157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:11.253577 19013 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16598372f92d463780b769327eb8c8ae in term 1.
I20260926 20:49:11.254973 19157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260926 20:49:11.289505 19169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:49:11.289821 19169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:49:11.292804 19013 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [term 1 FOLLOWER]: Refusing update from remote peer 16598372f92d463780b769327eb8c8ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:11.293614 18940 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [term 1 FOLLOWER]: Refusing update from remote peer 16598372f92d463780b769327eb8c8ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:11.294564 19126 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:11.294895 19154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:11.333276 19156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16598372f92d463780b769327eb8c8ae. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.333774 19156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.344563 19156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.344862 19156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.354444 19140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16598372f92d463780b769327eb8c8ae. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.354920 19140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.361094 19174 mvcc.cc:205] Tried to move back new op lower bound from 7333706757258375168 to 7333706757034119168. Current Snapshot: MvccSnapshot[applied={T|T < 7333706757258375168}]
I20260926 20:49:11.363133 19126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.363133 19154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.363512 19154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:11.363513 19126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:11.370764 19140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16598372f92d463780b769327eb8c8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130195e1c55a467ea2a8b2cb42692506" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42703 } } peers { permanent_uuid: "4932e27c4d2c4d0cb9ff548b255edfa0" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 33101 } } peers { permanent_uuid: "16598372f92d463780b769327eb8c8ae" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 45121 } } }
I20260926 20:49:11.371095 19140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:49:11.371630 19157 catalog_manager.cc:1347] Generated new cluster ID: 0f95c73b6e6a4064bec3a8d16905041f
I20260926 20:49:11.371744 19157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:49:11.416913 19157 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:49:11.419410 19157 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:49:11.450785 19157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16598372f92d463780b769327eb8c8ae: Generated new TSK 0
I20260926 20:49:11.452113 19157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:49:12.145526 19151 catalog_manager.cc:1259] Loaded cluster ID: 0f95c73b6e6a4064bec3a8d16905041f
I20260926 20:49:12.145656 19151 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: loading cluster ID for follower catalog manager: success
I20260926 20:49:12.148175 19151 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: acquiring CA information for follower catalog manager: success
I20260926 20:49:12.150027 19151 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 130195e1c55a467ea2a8b2cb42692506: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:49:12.292017 19169 catalog_manager.cc:1259] Loaded cluster ID: 0f95c73b6e6a4064bec3a8d16905041f
I20260926 20:49:12.292162 19169 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: loading cluster ID for follower catalog manager: success
I20260926 20:49:12.294761 19169 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: acquiring CA information for follower catalog manager: success
I20260926 20:49:12.296665 19169 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4932e27c4d2c4d0cb9ff548b255edfa0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:49:12.610740 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18898
I20260926 20:49:12.612367 18960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:12.722870 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18898
I20260926 20:49:12.744262 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 18972
I20260926 20:49:12.745441 19034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:12.854241 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 18972
I20260926 20:49:12.874943 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19046
I20260926 20:49:12.876171 19108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:12.986578 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19046
2026-09-26T20:49:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4857 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:49:13 dist-test-slave-z7j8 krb5kdc[19189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:49:13 dist-test-slave-z7j8 krb5kdc[19189](info): set up 2 sockets
Sep 26 20:49:13 dist-test-slave-z7j8 krb5kdc[19189](info): commencing operation
krb5kdc: starting...
W20260926 20:49:14.091261 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:49:14 dist-test-slave-z7j8 krb5kdc[19189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:49:14.340212 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42293
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:14.684101 19202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:14.684440 19202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:14.684598 19202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:14.692045 19202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:14.692167 19202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:14.692247 19202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:14.692337 19202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:14.692399 19202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:14.702687 19202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:14.704075 19202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:14.705473 19202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:14.718091 19210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:14.718047 19207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:14.718132 19208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:14.718982 19202 server_base.cc:1034] running on GCE node
I20260926 20:49:14.719857 19202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:14.721443 19202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:14.722752 19202 hybrid_clock.cc:648] HybridClock initialized: now 1790455754722658 us; error 52 us; skew 500 ppm
I20260926 20:49:14.723135 19202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:14 dist-test-slave-z7j8 krb5kdc[19189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455754, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:14.726544 19202 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:14.728327 19202 webserver.cc:460] Webserver started at http://127.5.78.62:39581/ using document root <none> and password file <none>
I20260926 20:49:14.729127 19202 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:14.729241 19202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:14.729640 19202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:14.733160 19202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "cd7432ee8efd47559a49b61ff7ef1ba4"
format_stamp: "Formatted at 2026-09-26 20:49:14 on dist-test-slave-z7j8"
server_key: "1898c6c59bc5ea49d59e8187565b83a1"
server_key_iv: "62707b2f8dd8a445c0676910bf9911f9"
server_key_version: "encryptionkey@0"
I20260926 20:49:14.734093 19202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "cd7432ee8efd47559a49b61ff7ef1ba4"
format_stamp: "Formatted at 2026-09-26 20:49:14 on dist-test-slave-z7j8"
server_key: "1898c6c59bc5ea49d59e8187565b83a1"
server_key_iv: "62707b2f8dd8a445c0676910bf9911f9"
server_key_version: "encryptionkey@0"
I20260926 20:49:14.740270 19202 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260926 20:49:14.744457 19217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:14.746138 19202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:49:14.746361 19202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "cd7432ee8efd47559a49b61ff7ef1ba4"
format_stamp: "Formatted at 2026-09-26 20:49:14 on dist-test-slave-z7j8"
server_key: "1898c6c59bc5ea49d59e8187565b83a1"
server_key_iv: "62707b2f8dd8a445c0676910bf9911f9"
server_key_version: "encryptionkey@0"
I20260926 20:49:14.746658 19202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:14.764782 19202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:14.770720 19202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:14.771036 19202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:14.790537 19202 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:42293
I20260926 20:49:14.790588 19268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:42293 every 8 connection(s)
I20260926 20:49:14.791970 19202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:14.794593 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19202
I20260926 20:49:14.794829 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:49:14.795668 5432 external_mini_cluster.cc:1428] Setting key 32b2ecefb1efc063ffb4abad7c71a98b
I20260926 20:49:14.796440 19269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:49:14 dist-test-slave-z7j8 krb5kdc[19189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:14.812678 19269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4: Bootstrap starting.
I20260926 20:49:14.817328 19269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:14.818646 19269 log.cc:824] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:14.822295 19269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4: No bootstrap required, opened a new log
I20260926 20:49:14.822546 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:14.797767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.5.78.62:42293)
0926 20:49:14.798714 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:14.798727 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:14.798774 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:49:14.800408 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:14.800432 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:14.800880 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:14.801462 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:14.803665 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:14.805128 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:14.807050 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:14.807769 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:14.807928 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:14.813160 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:14.813173 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:14.813195 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:14.813248 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:14.817493 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:14.818441 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:14.818452 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:14.818462 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:14.818706 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:14.819149 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:14.819155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:14.819159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:14.819866 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:14.820156 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:14.820656 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 20:49:14.821122 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":398,"threads_started":1}
I20260926 20:49:14.826959 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:49:14.831223 19269 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42293 } }
I20260926 20:49:14.831638 19269 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:14.831749 19269 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7432ee8efd47559a49b61ff7ef1ba4, State: Initialized, Role: FOLLOWER
I20260926 20:49:14.832417 19269 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42293 } }
I20260926 20:49:14.832656 19269 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:49:14.832818 19269 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:49:14.832998 19269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:14.835354 19269 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42293 } }
I20260926 20:49:14.835912 19269 leader_election.cc:302] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd7432ee8efd47559a49b61ff7ef1ba4; no voters:
I20260926 20:49:14.836560 19269 leader_election.cc:288] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:49:14.836717 19274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:14.837441 19274 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [term 1 LEADER]: Becoming Leader. State: Replica: cd7432ee8efd47559a49b61ff7ef1ba4, State: Running, Role: LEADER
I20260926 20:49:14.838045 19274 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42293 } }
I20260926 20:49:14.838778 19269 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:14.840968 19276 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7432ee8efd47559a49b61ff7ef1ba4. Latest consensus state: current_term: 1 leader_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42293 } } }
I20260926 20:49:14.841301 19275 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7432ee8efd47559a49b61ff7ef1ba4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42293 } } }
I20260926 20:49:14.841607 19276 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:14.841629 19275 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:14.843484 19282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:14.849093 19282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:49:14.858937 19282 catalog_manager.cc:1347] Generated new cluster ID: d7c28eadbcce43489de83858f79357f6
I20260926 20:49:14.859076 19282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:49:14.874275 19282 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:49:14.875689 19282 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:49:14.885722 19282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd7432ee8efd47559a49b61ff7ef1ba4: Generated new TSK 0
I20260926 20:49:14.886832 19282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:49:14.940363 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19202
I20260926 20:49:14.941681 19264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:15.038976 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19202
2026-09-26T20:49:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2050 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:49:15 dist-test-slave-z7j8 krb5kdc[19297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:49:15 dist-test-slave-z7j8 krb5kdc[19297](info): set up 2 sockets
Sep 26 20:49:15 dist-test-slave-z7j8 krb5kdc[19297](info): commencing operation
krb5kdc: starting...
W20260926 20:49:16.154886 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:49:16 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:49:16.396106 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42313
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:16.738186 19310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:16.738613 19310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:16.738715 19310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:16.746763 19310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:16.746910 19310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:16.746959 19310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:16.747010 19310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:16.747058 19310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:16.757483 19310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:16.758925 19310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:16.760234 19310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:16.773299 19315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:16.773332 19318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:16.773221 19316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:16.774437 19310 server_base.cc:1034] running on GCE node
I20260926 20:49:16.775664 19310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:16.778029 19310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:16.779379 19310 hybrid_clock.cc:648] HybridClock initialized: now 1790455756779295 us; error 80 us; skew 500 ppm
I20260926 20:49:16.779793 19310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:16 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455756, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:16.783396 19310 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:16.785058 19310 webserver.cc:460] Webserver started at http://127.5.78.62:40131/ using document root <none> and password file <none>
I20260926 20:49:16.785801 19310 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:16.785918 19310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:16.786180 19310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:16.789578 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "55feb692c1004dbfa2c14a5ac3fe8b69"
format_stamp: "Formatted at 2026-09-26 20:49:16 on dist-test-slave-z7j8"
server_key: "a2f9fa26c0e75f74079ce5146388c06a"
server_key_iv: "663d15ed6ba392d8e9726b4a6632aa9f"
server_key_version: "encryptionkey@0"
I20260926 20:49:16.790373 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "55feb692c1004dbfa2c14a5ac3fe8b69"
format_stamp: "Formatted at 2026-09-26 20:49:16 on dist-test-slave-z7j8"
server_key: "a2f9fa26c0e75f74079ce5146388c06a"
server_key_iv: "663d15ed6ba392d8e9726b4a6632aa9f"
server_key_version: "encryptionkey@0"
I20260926 20:49:16.796440 19310 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 20:49:16.800985 19325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:16.802950 19310 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 20:49:16.803153 19310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "55feb692c1004dbfa2c14a5ac3fe8b69"
format_stamp: "Formatted at 2026-09-26 20:49:16 on dist-test-slave-z7j8"
server_key: "a2f9fa26c0e75f74079ce5146388c06a"
server_key_iv: "663d15ed6ba392d8e9726b4a6632aa9f"
server_key_version: "encryptionkey@0"
I20260926 20:49:16.803359 19310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:16.819865 19310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:16.824847 19310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:16.825117 19310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:16.845454 19310 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:42313
I20260926 20:49:16.845484 19376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:42313 every 8 connection(s)
I20260926 20:49:16.847043 19310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:16.850699 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19310
I20260926 20:49:16.850927 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:49:16.851387 19377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:16.851653 5432 external_mini_cluster.cc:1428] Setting key 88d3d00ceacd755e2db6cf3e49a2ea40
I20260926 20:49:16.867208 19377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap starting.
Sep 26 20:49:16 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:16.870945 19377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:16.872128 19377 log.cc:824] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:16.875331 19377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: No bootstrap required, opened a new log
I20260926 20:49:16.877221 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:16.853644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58838 (local address 127.5.78.62:42313)
0926 20:49:16.854565 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:16.854578 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:16.854620 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:49:16.856268 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:16.856291 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:16.856702 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:16.857066 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:16.859013 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:16.860345 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:16.862349 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:16.863328 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:16.863504 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:16.868810 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:16.868823 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:16.868839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:16.868891 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:16.872623 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:16.873538 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:16.873546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:16.873557 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:16.873747 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:16.874188 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:16.874193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:16.874198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:16.874859 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:16.875212 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:16.875662 (+ 450us) server_negotiation.cc:299] Negotiation successful
0926 20:49:16.876153 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":386,"threads_started":1}
I20260926 20:49:16.884223 19377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:16.884687 19377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:16.884814 19377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55feb692c1004dbfa2c14a5ac3fe8b69, State: Initialized, Role: FOLLOWER
I20260926 20:49:16.885443 19377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:16.885735 19377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:49:16.885896 19377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:49:16.886086 19377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:16.888868 19377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:16.889498 19377 leader_election.cc:302] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55feb692c1004dbfa2c14a5ac3fe8b69; no voters:
I20260926 20:49:16.890213 19377 leader_election.cc:288] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:49:16.890434 19382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:16.891703 19382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 LEADER]: Becoming Leader. State: Replica: 55feb692c1004dbfa2c14a5ac3fe8b69, State: Running, Role: LEADER
I20260926 20:49:16.892283 19382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:16.892702 19377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:16.895655 19383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } } }
I20260926 20:49:16.895628 19384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55feb692c1004dbfa2c14a5ac3fe8b69. Latest consensus state: current_term: 1 leader_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } } }
I20260926 20:49:16.896104 19383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:16.896138 19384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:16.897258 19391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:16.905691 19391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:49:16.918164 19391 catalog_manager.cc:1347] Generated new cluster ID: 85aaf1fe593947aa9d6db44308190fc6
I20260926 20:49:16.918380 19391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:49:16.935729 19391 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:49:16.937798 19391 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:49:16.951303 19391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Generated new TSK 0
I20260926 20:49:16.952476 19391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:49:17.017781 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42313
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:17.365275 19403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:17.365593 19403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:17.365767 19403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:17.373359 19403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:17.373493 19403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:17.373698 19403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:49:17.384122 19403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:17.385522 19403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:17.387025 19403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:17.401422 19408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:17.401360 19411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:17.401396 19409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:17.402877 19403 server_base.cc:1034] running on GCE node
I20260926 20:49:17.403600 19403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:17.404618 19403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:17.405869 19403 hybrid_clock.cc:648] HybridClock initialized: now 1790455757405822 us; error 41 us; skew 500 ppm
I20260926 20:49:17.406243 19403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:17 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455757, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:17.409886 19403 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:49:17.411643 19403 webserver.cc:460] Webserver started at http://127.5.78.1:42177/ using document root <none> and password file <none>
I20260926 20:49:17.412410 19403 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:17.412555 19403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:17.412865 19403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:17.415910 19403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "57f4277b6cfc48938da1a5e77f566889"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "42e573978bdaf0614b6dc0640e74a5f2"
server_key_iv: "3fa0094ac7f7d4d995822f85bbe3c7e9"
server_key_version: "encryptionkey@0"
I20260926 20:49:17.416714 19403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "57f4277b6cfc48938da1a5e77f566889"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "42e573978bdaf0614b6dc0640e74a5f2"
server_key_iv: "3fa0094ac7f7d4d995822f85bbe3c7e9"
server_key_version: "encryptionkey@0"
I20260926 20:49:17.422359 19403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:49:17.426599 19418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:17.428138 19403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 20:49:17.428393 19403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "57f4277b6cfc48938da1a5e77f566889"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "42e573978bdaf0614b6dc0640e74a5f2"
server_key_iv: "3fa0094ac7f7d4d995822f85bbe3c7e9"
server_key_version: "encryptionkey@0"
I20260926 20:49:17.428637 19403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:17.446376 19403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:17.451179 19403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:17.451511 19403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:17.452545 19403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:17.454051 19403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:17.454169 19403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:17.454329 19403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:17.454402 19403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:17.489144 19403 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:39437
I20260926 20:49:17.489238 19530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:39437 every 8 connection(s)
I20260926 20:49:17.490437 19403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:49:17.492457 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19403
I20260926 20:49:17.492671 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:49:17.493446 5432 external_mini_cluster.cc:1428] Setting key 68cf59bda1f0da4b6147ea4e245e8fd8
Sep 26 20:49:17 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455757, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:17.516711 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:17.493417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:52835 (local address 127.5.78.62:42313)
0926 20:49:17.493913 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:17.493920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:17.495073 (+ 1153us) server_negotiation.cc:407] Connection header received
0926 20:49:17.497766 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:17.497770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:17.497836 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:17.497930 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:17.500118 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:17.500841 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:17.503493 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:17.504027 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:17.504180 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:17.508627 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:17.508637 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:17.508644 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:17.508697 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:17.512583 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:17.513568 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:17.513579 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:17.513585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:17.513765 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:17.514281 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:17.514289 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:17.514296 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:17.514758 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:17.514917 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:17.515907 (+ 990us) server_negotiation.cc:299] Negotiation successful
0926 20:49:17.516199 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260926 20:49:17.517436 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:17.494025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:52835)
0926 20:49:17.494904 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:17.494946 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:17.497444 (+ 2498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:17.498225 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:17.498240 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:17.499083 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:17.499916 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:17.499944 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:17.501029 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:17.501037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:17.503304 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:17.503325 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:17.504278 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:17.504284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:17.504507 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:17.505324 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:17.505355 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:17.508291 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:17.512801 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:17.512811 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:17.512828 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:17.513387 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:17.513904 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:17.513908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:17.513911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:17.514141 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:17.514935 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:17.514943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:17.515455 (+ 512us) client_negotiation.cc:769] Sending connection context
0926 20:49:17.515747 (+ 292us) client_negotiation.cc:240] Negotiation successful
0926 20:49:17.516210 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":365,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:49:17.519289 19531 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42313
I20260926 20:49:17.519629 19531 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:17.520413 19531 heartbeater.cc:511] Master 127.5.78.62:42313 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:49:17.522758 19342 ts_manager.cc:194] Registered new tserver with Master: 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437)
I20260926 20:49:17.524314 19342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:52835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:49:17.551209 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42313
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:17.894500 19537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:17.894817 19537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:17.894991 19537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:17.902700 19537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:17.902819 19537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:17.903028 19537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:49:17.913214 19537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:17.914535 19537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:17.915830 19537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:17.929149 19543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:17.929267 19545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:17.929239 19542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:17.930284 19537 server_base.cc:1034] running on GCE node
I20260926 20:49:17.930991 19537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:17.932147 19537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:17.933451 19537 hybrid_clock.cc:648] HybridClock initialized: now 1790455757933397 us; error 48 us; skew 500 ppm
I20260926 20:49:17.933871 19537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:17 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455757, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:17.937474 19537 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:49:17.939177 19537 webserver.cc:460] Webserver started at http://127.5.78.2:36713/ using document root <none> and password file <none>
I20260926 20:49:17.939935 19537 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:17.940092 19537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:17.940421 19537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:17.943542 19537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "984d37fcef264aac926f0d5847980591"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "f6156041503d28891b3e279c1ae89def"
server_key_iv: "77bb9ce09f95abfc2eb95dae20376fb0"
server_key_version: "encryptionkey@0"
I20260926 20:49:17.944324 19537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "984d37fcef264aac926f0d5847980591"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "f6156041503d28891b3e279c1ae89def"
server_key_iv: "77bb9ce09f95abfc2eb95dae20376fb0"
server_key_version: "encryptionkey@0"
I20260926 20:49:17.949810 19537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:49:17.953548 19552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:17.955226 19537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:49:17.955436 19537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "984d37fcef264aac926f0d5847980591"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "f6156041503d28891b3e279c1ae89def"
server_key_iv: "77bb9ce09f95abfc2eb95dae20376fb0"
server_key_version: "encryptionkey@0"
I20260926 20:49:17.955660 19537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:17.981134 19537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:17.985654 19537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:17.985975 19537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:17.987105 19537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:17.988572 19537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:17.988680 19537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:17.988829 19537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:17.988906 19537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:18.025544 19537 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:37499
I20260926 20:49:18.025736 19664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:37499 every 8 connection(s)
I20260926 20:49:18.027019 19537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:49:18.035312 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19537
I20260926 20:49:18.035534 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:49:18.035985 5432 external_mini_cluster.cc:1428] Setting key dc3f4a6b7a1702a331140db630c2b7c5
Sep 26 20:49:18 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455757, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:18.049973 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.029848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:46175 (local address 127.5.78.62:42313)
0926 20:49:18.030311 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.030315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.031408 (+ 1093us) server_negotiation.cc:407] Connection header received
0926 20:49:18.033358 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.033387 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.033482 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.033603 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:18.035485 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.036326 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.038186 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.038678 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.038814 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.043009 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:18.043018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:18.043027 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:18.043076 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:18.046497 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.047527 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.047532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.047534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.047667 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.048169 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.048172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.048174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.048426 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:18.048537 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.049495 (+ 958us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.049683 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":235,"threads_started":1}
I20260926 20:49:18.051621 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.030507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:46175)
0926 20:49:18.031229 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.031267 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.033067 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.033863 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.033872 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.034399 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:18.035281 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.035301 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.036498 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.036502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.038034 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.038051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.038923 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.038931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.039160 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.039903 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:18.039926 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:18.042619 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:18.046662 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:18.046672 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:18.046691 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:18.047394 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:18.047776 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:18.047783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:18.047789 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:18.048053 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:18.048558 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:18.048569 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:18.049055 (+ 486us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.049398 (+ 343us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.050026 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":275,"threads_started":1}
I20260926 20:49:18.053328 19665 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42313
I20260926 20:49:18.053679 19665 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:18.054651 19665 heartbeater.cc:511] Master 127.5.78.62:42313 requested a full tablet report, sending...
I20260926 20:49:18.056521 19342 ts_manager.cc:194] Registered new tserver with Master: 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499)
I20260926 20:49:18.057531 19342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:46175
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:49:18.098452 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42313
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:18.432763 19671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:18.433110 19671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:18.433281 19671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:18.440869 19671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:18.441036 19671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:18.441233 19671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:49:18.451612 19671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:18.453167 19671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:18.454646 19671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:18.468791 19676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:18.468883 19677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:18.469430 19671 server_base.cc:1034] running on GCE node
W20260926 20:49:18.469602 19679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:18.470335 19671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:18.471436 19671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:18.472707 19671 hybrid_clock.cc:648] HybridClock initialized: now 1790455758472651 us; error 48 us; skew 500 ppm
I20260926 20:49:18.473114 19671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:18 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455758, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:18.476825 19671 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:49:18.478689 19671 webserver.cc:460] Webserver started at http://127.5.78.3:34487/ using document root <none> and password file <none>
I20260926 20:49:18.479488 19671 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:18.479648 19671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:18.479974 19671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:18.483325 19671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "1fe5d9e1d41e4f14882309233f4a9267"
format_stamp: "Formatted at 2026-09-26 20:49:18 on dist-test-slave-z7j8"
server_key: "f147804131860b8237db8c9364587019"
server_key_iv: "8260838b66048723b985e8f18db62901"
server_key_version: "encryptionkey@0"
I20260926 20:49:18.484217 19671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "1fe5d9e1d41e4f14882309233f4a9267"
format_stamp: "Formatted at 2026-09-26 20:49:18 on dist-test-slave-z7j8"
server_key: "f147804131860b8237db8c9364587019"
server_key_iv: "8260838b66048723b985e8f18db62901"
server_key_version: "encryptionkey@0"
I20260926 20:49:18.490584 19671 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.006s
I20260926 20:49:18.495725 19686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:18.497561 19671 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260926 20:49:18.497800 19671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "1fe5d9e1d41e4f14882309233f4a9267"
format_stamp: "Formatted at 2026-09-26 20:49:18 on dist-test-slave-z7j8"
server_key: "f147804131860b8237db8c9364587019"
server_key_iv: "8260838b66048723b985e8f18db62901"
server_key_version: "encryptionkey@0"
I20260926 20:49:18.498050 19671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:18.520296 19671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:18.525309 19671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:18.525657 19671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:18.526853 19671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:18.527503 19531 heartbeater.cc:503] Master 127.5.78.62:42313 was elected leader, sending a full tablet report...
I20260926 20:49:18.528621 19671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:18.528736 19671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:18.528884 19671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:18.528956 19671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:18.566980 19671 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38807
I20260926 20:49:18.567051 19798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38807 every 8 connection(s)
I20260926 20:49:18.568480 19671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:49:18.573263 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19671
I20260926 20:49:18.573498 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:49:18.573983 5432 external_mini_cluster.cc:1428] Setting key db6daa6b1bac21a81df1a6b94e725a33
Sep 26 20:49:18 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455758, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:18.590385 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.571353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:36165 (local address 127.5.78.62:42313)
0926 20:49:18.571830 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.571834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.572891 (+ 1057us) server_negotiation.cc:407] Connection header received
0926 20:49:18.575209 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.575214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.575337 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.575477 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:18.577638 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.578508 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.580179 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.580612 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.580710 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.584262 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:18.584268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:18.584271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:18.584325 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:18.587012 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.587787 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.587791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.587793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.587912 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.588409 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.588412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.588415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.588680 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:18.588844 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.589766 (+ 922us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.590044 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":262,"threads_started":1}
I20260926 20:49:18.591440 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.571896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:36165)
0926 20:49:18.572698 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.572749 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.574904 (+ 2155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.575867 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.575882 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.576608 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:18.577377 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.577409 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.578711 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.578715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.580023 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.580036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.580872 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.580881 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.581124 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.581796 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:18.581822 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:18.584054 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:18.587148 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:18.587155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:18.587171 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:18.587640 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:18.588027 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:18.588035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:18.588041 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:18.588284 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:18.588811 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:18.588821 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:18.589321 (+ 500us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.589651 (+ 330us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.590215 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":305,"threads_started":1}
I20260926 20:49:18.592844 19799 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42313
I20260926 20:49:18.593168 19799 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:18.593938 19799 heartbeater.cc:511] Master 127.5.78.62:42313 requested a full tablet report, sending...
I20260926 20:49:18.595388 19342 ts_manager.cc:194] Registered new tserver with Master: 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3:38807)
I20260926 20:49:18.596205 19342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:36165
I20260926 20:49:18.602706 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:49:18.621035 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.607330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.5.78.62:42313)
0926 20:49:18.607588 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.607595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.607919 (+ 324us) server_negotiation.cc:407] Connection header received
0926 20:49:18.608121 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.608125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.608204 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.608299 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:18.609835 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.610691 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.612085 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.612564 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.612690 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.615348 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:18.615353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:18.615357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:18.615394 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:18.618168 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.619038 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.619043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.619045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.619200 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.619562 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.619565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.619567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.619922 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:18.620110 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.620433 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.620627 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:49:18.625752 19342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:49:18.632885 19342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:49:18.657703 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.646291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.5.78.2:37499)
0926 20:49:18.646903 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.646912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.648217 (+ 1305us) server_negotiation.cc:407] Connection header received
0926 20:49:18.648711 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.648720 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.649020 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.649236 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:18.650509 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.652126 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.654963 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.656387 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.656535 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.656770 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.657159 (+ 389us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.657342 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":1582,"threads_started":1}
I20260926 20:49:18.657858 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.646233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37499 (local address 127.0.0.1:59892)
0926 20:49:18.648048 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.648099 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.648404 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.649260 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.649270 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.649297 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:18.650271 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.650286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.652280 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.652286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.654763 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.654782 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.656641 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.656649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.656871 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.656935 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.657087 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.657288 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1530,"mutex_wait_us":559,"thread_start_us":1945,"threads_started":1}
I20260926 20:49:18.658861 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.646948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:39437 (local address 127.0.0.1:42138)
0926 20:49:18.648607 (+ 1659us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.648627 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.648771 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.649571 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.649577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.649600 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:18.650072 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.650088 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.653470 (+ 3382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.653478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.655794 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.655812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.657912 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.657920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.658141 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.658175 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.658265 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.658430 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1517,"mutex_wait_us":3,"thread_start_us":659,"threads_started":1}
I20260926 20:49:18.659729 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.646682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.5.78.1:39437)
0926 20:49:18.647460 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.647471 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.648741 (+ 1270us) server_negotiation.cc:407] Connection header received
0926 20:49:18.648942 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.648947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.649117 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.649312 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:18.651695 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.653316 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.655971 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.657352 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.657494 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.657741 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.659135 (+ 1394us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.659344 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"spinlock_wait_cycles":26496,"thread_start_us":495,"threads_started":1}
I20260926 20:49:18.661279 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.649663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38807 (local address 127.0.0.1:41038)
0926 20:49:18.651965 (+ 2302us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.651983 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.652107 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.652829 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.652836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.652869 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:18.653304 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.653317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.655474 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.655481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.657479 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.657494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.660509 (+ 3015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.660514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.660630 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.660650 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.660823 (+ 173us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.660961 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2187,"thread_start_us":2047,"threads_started":1}
I20260926 20:49:18.661491 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.649186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.5.78.3:38807)
0926 20:49:18.651566 (+ 2380us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.651575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.652101 (+ 526us) server_negotiation.cc:407] Connection header received
0926 20:49:18.652292 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.652302 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.652595 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.652833 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:18.653575 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.655311 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.657664 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.659975 (+ 2311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.660075 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.660329 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.660932 (+ 603us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.661126 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2197,"thread_start_us":2010,"threads_started":1}
I20260926 20:49:18.663040 19600 tablet_service.cc:1467] Processing CreateTablet for tablet b6364d90b9f945c2bf955094415783ef (DEFAULT_TABLE table=test-table [id=0e42b2b84ce8482f848dc62e64599274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:18.663441 19466 tablet_service.cc:1467] Processing CreateTablet for tablet b6364d90b9f945c2bf955094415783ef (DEFAULT_TABLE table=test-table [id=0e42b2b84ce8482f848dc62e64599274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:18.664747 19600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6364d90b9f945c2bf955094415783ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:18.664914 19466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6364d90b9f945c2bf955094415783ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:18.665422 19734 tablet_service.cc:1467] Processing CreateTablet for tablet b6364d90b9f945c2bf955094415783ef (DEFAULT_TABLE table=test-table [id=0e42b2b84ce8482f848dc62e64599274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:18.667029 19734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6364d90b9f945c2bf955094415783ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:18.676801 19818 tablet_bootstrap.cc:492] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Bootstrap starting.
I20260926 20:49:18.680953 19819 tablet_bootstrap.cc:492] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Bootstrap starting.
I20260926 20:49:18.681687 19818 tablet_bootstrap.cc:654] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:18.683125 19818 log.cc:824] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:18.683190 19820 tablet_bootstrap.cc:492] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Bootstrap starting.
I20260926 20:49:18.685092 19819 tablet_bootstrap.cc:654] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:18.686465 19819 log.cc:824] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:18.687170 19820 tablet_bootstrap.cc:654] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:18.687011 19818 tablet_bootstrap.cc:492] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: No bootstrap required, opened a new log
I20260926 20:49:18.688035 19818 ts_tablet_manager.cc:1397] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260926 20:49:18.688581 19820 log.cc:824] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:18.690233 19819 tablet_bootstrap.cc:492] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: No bootstrap required, opened a new log
I20260926 20:49:18.690577 19819 ts_tablet_manager.cc:1397] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 20:49:18.691483 19820 tablet_bootstrap.cc:492] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: No bootstrap required, opened a new log
I20260926 20:49:18.691771 19820 ts_tablet_manager.cc:1397] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260926 20:49:18.699697 19818 raft_consensus.cc:348] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.700274 19818 raft_consensus.cc:374] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:18.700418 19818 raft_consensus.cc:729] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f4277b6cfc48938da1a5e77f566889, State: Initialized, Role: FOLLOWER
I20260926 20:49:18.700752 19820 raft_consensus.cc:348] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.701239 19820 raft_consensus.cc:374] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:18.701360 19820 raft_consensus.cc:729] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5d9e1d41e4f14882309233f4a9267, State: Initialized, Role: FOLLOWER
I20260926 20:49:18.701191 19818 consensus_queue.cc:260] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.701673 19819 raft_consensus.cc:348] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.702040 19820 consensus_queue.cc:260] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.702294 19819 raft_consensus.cc:374] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:18.702490 19819 raft_consensus.cc:729] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984d37fcef264aac926f0d5847980591, State: Initialized, Role: FOLLOWER
I20260926 20:49:18.703356 19819 consensus_queue.cc:260] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.703610 19799 heartbeater.cc:503] Master 127.5.78.62:42313 was elected leader, sending a full tablet report...
I20260926 20:49:18.704484 19820 ts_tablet_manager.cc:1428] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 20:49:18.705260 19819 ts_tablet_manager.cc:1428] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260926 20:49:18.705341 19665 heartbeater.cc:503] Master 127.5.78.62:42313 was elected leader, sending a full tablet report...
I20260926 20:49:18.707762 19818 ts_tablet_manager.cc:1428] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Time spent starting tablet: real 0.020s user 0.012s sys 0.003s
I20260926 20:49:18.718703 19825 raft_consensus.cc:482] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:49:18.719072 19825 raft_consensus.cc:504] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.721071 19825 leader_election.cc:288] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499), 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437)
I20260926 20:49:18.728011 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.721422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37499 (local address 127.5.78.3:50367)
0926 20:49:18.721860 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.721886 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.722079 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.722509 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.722514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.722596 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:18.722821 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.722838 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.724208 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.724216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.725855 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.725871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.727143 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.727149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.727381 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.727418 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.727532 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.727689 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":1,"spinlock_wait_cycles":25856}
I20260926 20:49:18.728107 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.721582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:50367 (local address 127.5.78.2:37499)
0926 20:49:18.721825 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.721831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.721993 (+ 162us) server_negotiation.cc:407] Connection header received
0926 20:49:18.722267 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.722272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.722335 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.722474 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:18.722992 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.724016 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.726041 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.726964 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.727118 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.727214 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.727732 (+ 518us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.727875 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:49:18.728412 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.721715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:53467 (local address 127.5.78.1:39437)
0926 20:49:18.721934 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.721938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.722478 (+ 540us) server_negotiation.cc:407] Connection header received
0926 20:49:18.722689 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.722693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.722752 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.722826 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:18.723410 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.724356 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.726145 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.727293 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.727459 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.727575 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.727961 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.728111 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:49:18.728995 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.721491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:39437 (local address 127.5.78.3:53467)
0926 20:49:18.722351 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:18.722365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:18.722543 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:18.722886 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:18.722893 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:18.722926 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:18.723235 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.723243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.724513 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.724529 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.725969 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:18.725991 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.727530 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.727537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:18.727749 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.727783 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:49:18.728523 (+ 740us) client_negotiation.cc:240] Negotiation successful
0926 20:49:18.728658 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":3,"thread_start_us":473,"threads_started":1}
I20260926 20:49:18.729234 19620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6364d90b9f945c2bf955094415783ef" candidate_uuid: "1fe5d9e1d41e4f14882309233f4a9267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d37fcef264aac926f0d5847980591" is_pre_election: true
I20260926 20:49:18.729816 19620 raft_consensus.cc:2454] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5d9e1d41e4f14882309233f4a9267 in term 0.
I20260926 20:49:18.729923 19486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6364d90b9f945c2bf955094415783ef" candidate_uuid: "1fe5d9e1d41e4f14882309233f4a9267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f4277b6cfc48938da1a5e77f566889" is_pre_election: true
I20260926 20:49:18.730520 19486 raft_consensus.cc:2454] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5d9e1d41e4f14882309233f4a9267 in term 0.
I20260926 20:49:18.730554 19687 leader_election.cc:302] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5d9e1d41e4f14882309233f4a9267, 984d37fcef264aac926f0d5847980591; no voters:
I20260926 20:49:18.731009 19825 raft_consensus.cc:2781] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:49:18.731231 19825 raft_consensus.cc:482] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:49:18.731383 19825 raft_consensus.cc:3037] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:18.734072 19825 raft_consensus.cc:504] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
I20260926 20:49:18.734855 19825 leader_election.cc:288] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [CANDIDATE]: Term 1 election: Requested vote from peers 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499), 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437)
I20260926 20:49:18.735461 19486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6364d90b9f945c2bf955094415783ef" candidate_uuid: "1fe5d9e1d41e4f14882309233f4a9267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f4277b6cfc48938da1a5e77f566889"
I20260926 20:49:18.735496 19620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6364d90b9f945c2bf955094415783ef" candidate_uuid: "1fe5d9e1d41e4f14882309233f4a9267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d37fcef264aac926f0d5847980591"
I20260926 20:49:18.735697 19486 raft_consensus.cc:3037] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:18.735728 19620 raft_consensus.cc:3037] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:18.738162 19620 raft_consensus.cc:2454] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5d9e1d41e4f14882309233f4a9267 in term 1.
I20260926 20:49:18.738185 19486 raft_consensus.cc:2454] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5d9e1d41e4f14882309233f4a9267 in term 1.
I20260926 20:49:18.738729 19687 leader_election.cc:302] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5d9e1d41e4f14882309233f4a9267, 984d37fcef264aac926f0d5847980591; no voters:
I20260926 20:49:18.739118 19825 raft_consensus.cc:2781] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:18.739605 19825 raft_consensus.cc:686] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 LEADER]: Becoming Leader. State: Replica: 1fe5d9e1d41e4f14882309233f4a9267, State: Running, Role: LEADER
I20260926 20:49:18.740192 19825 consensus_queue.cc:237] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } }
W20260926 20:49:18.742882 19532 tablet.cc:2369] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:18.745329 19341 catalog_manager.cc:5495] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3). New cstate: current_term: 1 leader_uuid: "1fe5d9e1d41e4f14882309233f4a9267" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260926 20:49:18.779459 19666 tablet.cc:2369] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:18.787755 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.780497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.5.78.3:38807)
0926 20:49:18.780726 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.780733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.780755 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:49:18.780943 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.780948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.781061 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.781236 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:49:18.781658 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.782869 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.785213 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.785727 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.785878 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.786519 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:49:18.787058 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:49:18.787234 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.787329 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.787445 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:49:18.796279 19620 raft_consensus.cc:1261] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Refusing update from remote peer 1fe5d9e1d41e4f14882309233f4a9267: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:18.796278 19486 raft_consensus.cc:1261] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Refusing update from remote peer 1fe5d9e1d41e4f14882309233f4a9267: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:18.797372 19828 consensus_queue.cc:1035] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:18.797777 19825 consensus_queue.cc:1035] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:18.814460 19831 mvcc.cc:205] Tried to move back new op lower bound from 7333706788018946048 to 7333706787804536832. Current Snapshot: MvccSnapshot[applied={T|T < 7333706788018946048 or (T in {7333706788018946048})}]
I20260926 20:49:18.816548 19832 mvcc.cc:205] Tried to move back new op lower bound from 7333706788018946048 to 7333706787804536832. Current Snapshot: MvccSnapshot[applied={T|T < 7333706788018946048}]
I20260926 20:49:18.826149 19833 mvcc.cc:205] Tried to move back new op lower bound from 7333706788018946048 to 7333706787804536832. Current Snapshot: MvccSnapshot[applied={T|T < 7333706788018946048}]
I20260926 20:49:18.829990 19342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:18.830331 19342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:18.834352 19342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Sending DeleteTablet for 3 replicas of tablet b6364d90b9f945c2bf955094415783ef
I20260926 20:49:18.835608 19466 tablet_service.cc:1514] Processing DeleteTablet for tablet b6364d90b9f945c2bf955094415783ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:18 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:42138
I20260926 20:49:18.835871 19734 tablet_service.cc:1514] Processing DeleteTablet for tablet b6364d90b9f945c2bf955094415783ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:18 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:41038
I20260926 20:49:18.836134 19600 tablet_service.cc:1514] Processing DeleteTablet for tablet b6364d90b9f945c2bf955094415783ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:18 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:59892
I20260926 20:49:18.837119 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19403
I20260926 20:49:18.863343 19841 tablet_replica.cc:330] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: stopping tablet replica
I20260926 20:49:18.863950 19841 raft_consensus.cc:2229] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:18.866011 19841 raft_consensus.cc:2258] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:18.860478 19843 tablet_replica.cc:330] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: stopping tablet replica
I20260926 20:49:18.867108 19843 raft_consensus.cc:2229] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:18.867722 19843 raft_consensus.cc:2258] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:18.868620 19842 tablet_replica.cc:330] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: stopping tablet replica
I20260926 20:49:18.868858 19841 ts_tablet_manager.cc:1905] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:18.869163 19842 raft_consensus.cc:2229] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:49:18.869882 19842 raft_consensus.cc:2258] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:18.870383 19843 ts_tablet_manager.cc:1905] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:18.876645 19841 ts_tablet_manager.cc:1918] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:18.876879 19841 log.cc:1196] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/b6364d90b9f945c2bf955094415783ef
I20260926 20:49:18.877501 19841 ts_tablet_manager.cc:1939] T b6364d90b9f945c2bf955094415783ef P 57f4277b6cfc48938da1a5e77f566889: Deleting consensus metadata
I20260926 20:49:18.877669 19843 ts_tablet_manager.cc:1918] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:18.877898 19843 log.cc:1196] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/b6364d90b9f945c2bf955094415783ef
I20260926 20:49:18.878522 19843 ts_tablet_manager.cc:1939] T b6364d90b9f945c2bf955094415783ef P 984d37fcef264aac926f0d5847980591: Deleting consensus metadata
I20260926 20:49:18.879779 19327 catalog_manager.cc:4841] TS 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437): tablet b6364d90b9f945c2bf955094415783ef (table test-table [id=0e42b2b84ce8482f848dc62e64599274]) successfully deleted
I20260926 20:49:18.880376 19842 ts_tablet_manager.cc:1905] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:18.880390 19326 catalog_manager.cc:4841] TS 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499): tablet b6364d90b9f945c2bf955094415783ef (table test-table [id=0e42b2b84ce8482f848dc62e64599274]) successfully deleted
I20260926 20:49:18.886919 19842 ts_tablet_manager.cc:1918] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:18.887169 19842 log.cc:1196] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/b6364d90b9f945c2bf955094415783ef
I20260926 20:49:18.887681 19842 ts_tablet_manager.cc:1939] T b6364d90b9f945c2bf955094415783ef P 1fe5d9e1d41e4f14882309233f4a9267: Deleting consensus metadata
Sep 26 20:49:18 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:49:18.889348 19326 catalog_manager.cc:4841] TS 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3:38807): tablet b6364d90b9f945c2bf955094415783ef (table test-table [id=0e42b2b84ce8482f848dc62e64599274]) successfully deleted
I20260926 20:49:18.896427 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:18.838560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.5.78.1:39437)
0926 20:49:18.838798 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:18.838808 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:18.860863 (+ 22055us) server_negotiation.cc:407] Connection header received
0926 20:49:18.861142 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:18.861149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:18.861242 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:18.861313 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:18.866651 (+ 5338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.868313 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.882711 (+ 14398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:18.883395 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:18.883567 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:18.889431 (+ 5864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:18.889449 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:18.889469 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:18.889537 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:18.892926 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.893975 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.893983 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.893985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.894136 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:18.894717 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:18.894722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:18.894724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:18.895138 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:18.895375 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:18.895806 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 20:49:18.895994 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:49:18.897159 19526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:18.991807 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19403
I20260926 20:49:19.010641 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19537
Sep 26 20:49:19 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:49:19.030306 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:19.011658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.5.78.2:37499)
0926 20:49:19.011885 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:19.011891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:19.011919 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:49:19.011991 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:19.011997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:19.012090 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:19.012202 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:19.013655 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:19.014966 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:19.016376 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:19.017026 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:19.017167 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:19.022113 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:19.022136 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:19.022158 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:19.022213 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:19.026614 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:19.027695 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:19.027708 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:19.027711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:19.027893 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:19.028376 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:19.028382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:19.028385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:19.028971 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:19.029301 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:19.029633 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 20:49:19.029914 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:49:19.031152 19660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:19.133848 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19537
I20260926 20:49:19.157471 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19671
Sep 26 20:49:19 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:49:19.177198 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:19.158364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.5.78.3:38807)
0926 20:49:19.158616 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:19.158631 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:19.158883 (+ 252us) server_negotiation.cc:407] Connection header received
0926 20:49:19.159034 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:19.159040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:19.159154 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:19.159275 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:19.160635 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:19.161977 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:19.163389 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:19.164092 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:19.164263 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:19.169304 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:19.169326 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:19.169349 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:19.169412 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:19.173764 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:19.174722 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:19.174731 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:19.174734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:19.174910 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:19.175439 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:19.175469 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:19.175472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:19.175891 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:19.176162 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:19.176615 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 20:49:19.176802 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 20:49:19.178009 19794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:19.305789 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19671
I20260926 20:49:19.323007 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19310
I20260926 20:49:19.324255 19372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:19.435595 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19310
I20260926 20:49:19.456950 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42313
--webserver_interface=127.5.78.62
--webserver_port=40131
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:19.803776 19849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:19.804091 19849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:19.804278 19849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:19.812157 19849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:19.812294 19849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:19.812381 19849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:19.812479 19849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:19.812582 19849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:19.822918 19849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=40131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:19.824308 19849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:19.825678 19849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:19.849272 19856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:19.849398 19855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:19.849359 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:19.850231 19849 server_base.cc:1034] running on GCE node
I20260926 20:49:19.851249 19849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:19.853024 19849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:19.854259 19849 hybrid_clock.cc:648] HybridClock initialized: now 1790455759854209 us; error 48 us; skew 500 ppm
I20260926 20:49:19.854790 19849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:19 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455759, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:19.858783 19849 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:19.860724 19849 webserver.cc:460] Webserver started at http://127.5.78.62:40131/ using document root <none> and password file <none>
I20260926 20:49:19.861559 19849 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:19.861716 19849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:19.868461 19849 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:49:19.871896 19865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:19.873502 19849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:49:19.873720 19849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "55feb692c1004dbfa2c14a5ac3fe8b69"
format_stamp: "Formatted at 2026-09-26 20:49:16 on dist-test-slave-z7j8"
server_key: "a2f9fa26c0e75f74079ce5146388c06a"
server_key_iv: "663d15ed6ba392d8e9726b4a6632aa9f"
server_key_version: "encryptionkey@0"
I20260926 20:49:19.874938 19849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:19.891736 19849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:19.896616 19849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:19.896961 19849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:19.917222 19849 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:42313
I20260926 20:49:19.917243 19916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:42313 every 8 connection(s)
I20260926 20:49:19.918957 19849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:19.921306 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19849
I20260926 20:49:19.922309 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:39437
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:42313
--webserver_port=42177
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:49:19.929314 19917 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:49:19.933357 19917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap starting.
I20260926 20:49:19.953893 19917 log.cc:824] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:19.966172 19917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:49:19.966805 19917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap complete.
I20260926 20:49:19.976591 19917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:19.977581 19917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55feb692c1004dbfa2c14a5ac3fe8b69, State: Initialized, Role: FOLLOWER
I20260926 20:49:19.978497 19917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:19.978770 19917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:49:19.978904 19917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:49:19.979115 19917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:49:19.983647 19917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:19.984274 19917 leader_election.cc:302] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55feb692c1004dbfa2c14a5ac3fe8b69; no voters:
I20260926 20:49:19.985001 19917 leader_election.cc:288] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:49:19.985148 19921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:49:19.986356 19921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 LEADER]: Becoming Leader. State: Replica: 55feb692c1004dbfa2c14a5ac3fe8b69, State: Running, Role: LEADER
I20260926 20:49:19.987116 19921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:19.987574 19917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:19.990878 19922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } } }
I20260926 20:49:19.991282 19922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:19.991586 19923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55feb692c1004dbfa2c14a5ac3fe8b69. Latest consensus state: current_term: 2 leader_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } } }
I20260926 20:49:19.991887 19923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:20.005280 19928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:20.013854 19928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:49:20.015492 19928 catalog_manager.cc:1259] Loaded cluster ID: 85aaf1fe593947aa9d6db44308190fc6
I20260926 20:49:20.015682 19928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:49:20.019094 19928 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 20:49:20.036955 19928 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 20:49:20.047081 19928 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:49:20.048949 19928 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Loaded TSK: 0
I20260926 20:49:20.050870 19928 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:49:20.297374 19919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:20.297726 19919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:20.297891 19919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:20.305891 19919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:20.306058 19919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:20.306254 19919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:49:20.316998 19919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:39437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=42177
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:20.318463 19919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:20.319913 19919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:20.336108 19945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:20.336393 19947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:20.336956 19944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:20.340780 19919 server_base.cc:1034] running on GCE node
I20260926 20:49:20.341610 19919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:20.342799 19919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:20.344098 19919 hybrid_clock.cc:648] HybridClock initialized: now 1790455760344006 us; error 78 us; skew 500 ppm
I20260926 20:49:20.344515 19919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:20 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455760, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:20.353226 19919 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:49:20.355274 19919 webserver.cc:460] Webserver started at http://127.5.78.1:42177/ using document root <none> and password file <none>
I20260926 20:49:20.356066 19919 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:20.356257 19919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:20.362186 19919 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:49:20.365429 19954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:20.367250 19919 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:20.367451 19919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "57f4277b6cfc48938da1a5e77f566889"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "42e573978bdaf0614b6dc0640e74a5f2"
server_key_iv: "3fa0094ac7f7d4d995822f85bbe3c7e9"
server_key_version: "encryptionkey@0"
I20260926 20:49:20.368541 19919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:20.386314 19919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:20.391831 19919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:20.392174 19919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:20.393356 19919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:20.394946 19919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:20.395041 19919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:20.395231 19919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:20.395328 19919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:20.432130 19919 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:39437
I20260926 20:49:20.432184 20066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:39437 every 8 connection(s)
I20260926 20:49:20.433683 19919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:49:20.439942 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 19919
I20260926 20:49:20.440742 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:37499
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:42313
--webserver_port=36713
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:49:20 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455760, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:20.463207 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:20.436784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:42185)
0926 20:49:20.437758 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:20.437811 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:20.440115 (+ 2304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:20.442040 (+ 1925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:20.442056 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:20.442962 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:20.443895 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:20.443927 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.445807 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.445815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:20.448053 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:20.448071 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.449828 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.449847 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:20.450060 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:20.450896 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:20.450920 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:20.453206 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:20.458487 (+ 5281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:20.458495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:20.458509 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:20.459010 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:20.459623 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:20.459640 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:20.459643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:20.459840 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:20.460968 (+ 1128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:20.460974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:20.461325 (+ 351us) client_negotiation.cc:769] Sending connection context
0926 20:49:20.461640 (+ 315us) client_negotiation.cc:240] Negotiation successful
0926 20:49:20.462078 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":460,"threads_started":1}
I20260926 20:49:20.464032 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:20.436731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:42185 (local address 127.5.78.62:42313)
0926 20:49:20.437394 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:20.437407 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:20.438026 (+ 619us) server_negotiation.cc:407] Connection header received
0926 20:49:20.440891 (+ 2865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:20.440922 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:20.441389 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:20.441767 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:20.444244 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.445591 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.448264 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.449525 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.449784 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:20.453612 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:20.453626 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:20.453657 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:20.453717 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:20.458215 (+ 4498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:20.459205 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:20.459217 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:20.459227 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:20.459492 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:20.459978 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:20.459985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:20.459991 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:20.460796 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:20.461038 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:20.461958 (+ 920us) server_negotiation.cc:299] Negotiation successful
0926 20:49:20.462569 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":305,"threads_started":1}
I20260926 20:49:20.467319 20067 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42313
I20260926 20:49:20.467677 20067 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:20.468461 20067 heartbeater.cc:511] Master 127.5.78.62:42313 requested a full tablet report, sending...
I20260926 20:49:20.470940 19882 ts_manager.cc:194] Registered new tserver with Master: 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437)
I20260926 20:49:20.472654 19882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:42185
W20260926 20:49:20.794938 20071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:20.795313 20071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:20.795513 20071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:20.804468 20071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:20.804627 20071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:20.804756 20071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:49:20.815227 20071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:37499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=36713
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:20.816546 20071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:20.817905 20071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:20.832155 20076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:20.832219 20079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:20.832195 20077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:20.833086 20071 server_base.cc:1034] running on GCE node
I20260926 20:49:20.833765 20071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:20.834897 20071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:20.836135 20071 hybrid_clock.cc:648] HybridClock initialized: now 1790455760836068 us; error 71 us; skew 500 ppm
I20260926 20:49:20.836541 20071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:20 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455760, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:20.839988 20071 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:49:20.841763 20071 webserver.cc:460] Webserver started at http://127.5.78.2:36713/ using document root <none> and password file <none>
I20260926 20:49:20.842649 20071 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:20.842823 20071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:20.848731 20071 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:49:20.851958 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:20.853725 20071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:49:20.853955 20071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "984d37fcef264aac926f0d5847980591"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "f6156041503d28891b3e279c1ae89def"
server_key_iv: "77bb9ce09f95abfc2eb95dae20376fb0"
server_key_version: "encryptionkey@0"
I20260926 20:49:20.855142 20071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:20.868284 20071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:20.873118 20071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:20.873481 20071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:20.874820 20071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:20.876272 20071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:20.876394 20071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:20.876539 20071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:20.876631 20071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:20.912492 20071 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:37499
I20260926 20:49:20.912587 20198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:37499 every 8 connection(s)
I20260926 20:49:20.914065 20071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:49:20.918689 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20071
I20260926 20:49:20.919492 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:38807
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:42313
--webserver_port=34487
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:49:20 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455760, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:20.940337 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:20.917104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:51177 (local address 127.5.78.62:42313)
0926 20:49:20.917286 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:20.917291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:20.919051 (+ 1760us) server_negotiation.cc:407] Connection header received
0926 20:49:20.922359 (+ 3308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:20.922369 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:20.922550 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:20.922677 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:20.925270 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.926377 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.928483 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.929143 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.929308 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:20.933498 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:20.933504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:20.933507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:20.933546 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:20.936619 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:20.937550 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:20.937557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:20.937561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:20.937735 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:20.938270 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:20.938275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:20.938278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:20.938663 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:20.938827 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:20.939663 (+ 836us) server_negotiation.cc:299] Negotiation successful
0926 20:49:20.939939 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:49:20.941520 20201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:20.917693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:51177)
0926 20:49:20.918840 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:20.918907 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:20.921931 (+ 3024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:20.923060 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:20.923077 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:20.924011 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:20.924999 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:20.925033 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.926626 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.926635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:20.928286 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:20.928301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:20.929421 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:20.929428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:20.929688 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:20.930527 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:20.930553 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:20.933212 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:20.936838 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:20.936848 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:20.936875 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:20.937393 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:20.937879 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:20.937885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:20.937888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:20.938138 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:20.938828 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:20.938839 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:20.939237 (+ 398us) client_negotiation.cc:769] Sending connection context
0926 20:49:20.939547 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 20:49:20.940177 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":495,"threads_started":1}
I20260926 20:49:20.943284 20199 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42313
I20260926 20:49:20.943648 20199 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:20.944453 20199 heartbeater.cc:511] Master 127.5.78.62:42313 requested a full tablet report, sending...
I20260926 20:49:20.946034 19882 ts_manager.cc:194] Registered new tserver with Master: 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499)
I20260926 20:49:20.946933 19882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:51177
W20260926 20:49:21.266376 20202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:21.266788 20202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:21.266960 20202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:21.274474 20202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:21.274644 20202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:21.274827 20202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:49:21.285046 20202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:38807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=34487
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:21.286475 20202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:21.287964 20202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:21.301896 20207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:21.302001 20210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:21.301971 20208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:21.302801 20202 server_base.cc:1034] running on GCE node
I20260926 20:49:21.303503 20202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:21.304621 20202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:21.305843 20202 hybrid_clock.cc:648] HybridClock initialized: now 1790455761305765 us; error 70 us; skew 500 ppm
I20260926 20:49:21.306193 20202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:21 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455761, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:21.309654 20202 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:49:21.311594 20202 webserver.cc:460] Webserver started at http://127.5.78.3:34487/ using document root <none> and password file <none>
I20260926 20:49:21.312386 20202 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:21.312562 20202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:21.318204 20202 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:49:21.321362 20217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:21.323097 20202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:49:21.323314 20202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "1fe5d9e1d41e4f14882309233f4a9267"
format_stamp: "Formatted at 2026-09-26 20:49:18 on dist-test-slave-z7j8"
server_key: "f147804131860b8237db8c9364587019"
server_key_iv: "8260838b66048723b985e8f18db62901"
server_key_version: "encryptionkey@0"
I20260926 20:49:21.324433 20202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:21.344709 20202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:21.349567 20202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:21.349920 20202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:21.351179 20202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:21.352622 20202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:21.352754 20202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:21.352900 20202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:21.352998 20202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:21.388679 20202 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38807
I20260926 20:49:21.388805 20329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38807 every 8 connection(s)
I20260926 20:49:21.390249 20202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:49:21.392259 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20202
Sep 26 20:49:21 dist-test-slave-z7j8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455761, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:21.411145 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.393361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.5.78.62:42313)
0926 20:49:21.393626 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.393636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.395070 (+ 1434us) server_negotiation.cc:407] Connection header received
0926 20:49:21.395394 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.395398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.395463 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.395576 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:21.397639 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.398432 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.400453 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.400876 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.400988 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.404815 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:21.404821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:21.404824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:21.404863 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:21.407895 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.408762 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.408771 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.408776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.408931 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.409347 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.409352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.409357 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.409728 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:21.410020 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.410437 (+ 417us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.410701 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"spinlock_wait_cycles":7552}
I20260926 20:49:21.417136 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.393476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:52161 (local address 127.5.78.62:42313)
0926 20:49:21.395197 (+ 1721us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.395202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.396584 (+ 1382us) server_negotiation.cc:407] Connection header received
0926 20:49:21.399021 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.399026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.399102 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.399194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:21.401271 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.402226 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.405123 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.405864 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.406025 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.410339 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:21.410346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:21.410351 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:21.410448 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:21.413910 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.414761 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.414765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.414768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.414910 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.415407 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.415411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.415414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.415740 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:21.415911 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.416659 (+ 748us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.416829 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1612,"spinlock_wait_cycles":2048,"thread_start_us":1483,"threads_started":1}
I20260926 20:49:21.418193 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.394244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:52161)
0926 20:49:21.396346 (+ 2102us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.396426 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.398667 (+ 2241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.399546 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.399560 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.400343 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:21.401070 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.401098 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.402489 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.402497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.404947 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.404965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.406128 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.406134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.406349 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.407215 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:21.407244 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:21.410052 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:21.414062 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:21.414070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:21.414086 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:21.414620 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:21.415058 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:21.415062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:21.415064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:21.415263 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:21.415920 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:21.415927 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:21.416287 (+ 360us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.416561 (+ 274us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.416999 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1728,"thread_start_us":1409,"threads_started":1}
I20260926 20:49:21.420035 20330 heartbeater.cc:347] Connected to a master server at 127.5.78.62:42313
I20260926 20:49:21.420389 20330 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:21.421223 20330 heartbeater.cc:511] Master 127.5.78.62:42313 requested a full tablet report, sending...
I20260926 20:49:21.423018 19882 ts_manager.cc:194] Registered new tserver with Master: 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3:38807)
I20260926 20:49:21.423998 19882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:52161
I20260926 20:49:21.426810 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:49:21.434531 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.427871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.5.78.62:42313)
0926 20:49:21.428108 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.428114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.428499 (+ 385us) server_negotiation.cc:407] Connection header received
0926 20:49:21.428789 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.428794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.428957 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.429084 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:49:21.429566 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.430306 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.431905 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.432511 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.432664 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.433182 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:49:21.433897 (+ 715us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:49:21.434043 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.434128 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.434226 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:49:21.435750 19882 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:49:21.444269 19882 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:49:21.467859 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.457097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.5.78.3:38807)
0926 20:49:21.458959 (+ 1862us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.458970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.459006 (+ 36us) server_negotiation.cc:407] Connection header received
0926 20:49:21.459112 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.459119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.459402 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.460398 (+ 996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.460692 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.462350 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.465288 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.466632 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.466780 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.467042 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.467325 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.467494 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1659,"thread_start_us":1510,"threads_started":1}
I20260926 20:49:21.467905 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.456992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38807 (local address 127.0.0.1:41072)
0926 20:49:21.457947 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.458013 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.458225 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.459676 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.459685 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.459726 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.460197 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.460211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.462553 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.462558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.465098 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.465114 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.466844 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.466854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.467076 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.467129 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.467271 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.467492 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":1,"thread_start_us":405,"threads_started":1}
I20260926 20:49:21.469264 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.457368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:39437 (local address 127.0.0.1:42162)
0926 20:49:21.459710 (+ 2342us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.459727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.459860 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.461431 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.461435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.461455 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.461800 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.461812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.463851 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.463858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.465990 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.466006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.468485 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.468493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.468656 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.468689 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.468768 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.468889 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2208,"spinlock_wait_cycles":768,"thread_start_us":3508,"threads_started":1}
I20260926 20:49:21.469794 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.458009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42162 (local address 127.5.78.1:39437)
0926 20:49:21.459267 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.459274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.459994 (+ 720us) server_negotiation.cc:407] Connection header received
0926 20:49:21.460103 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.460110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.460357 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.460557 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.462011 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.463650 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.466167 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.467679 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.467796 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.469196 (+ 1400us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.469268 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.469455 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1106,"thread_start_us":979,"threads_started":1}
I20260926 20:49:21.470273 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.458302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37499 (local address 127.0.0.1:59904)
0926 20:49:21.461185 (+ 2883us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.461203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.461307 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.463234 (+ 1927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.463239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.463263 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.463755 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.463774 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.465448 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.465452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.467398 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.467415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.469480 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.469486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.469698 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.469728 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.469820 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.469966 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2751,"thread_start_us":1124,"threads_started":1}
I20260926 20:49:21.470564 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.458805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.5.78.2:37499)
0926 20:49:21.462764 (+ 3959us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.462774 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.462802 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:49:21.462881 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.462885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.463058 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.463249 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.463913 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.465279 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.467588 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.469001 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.469132 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.469323 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.470033 (+ 710us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.470204 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3758,"thread_start_us":3659,"threads_started":1}
I20260926 20:49:21.472337 20265 tablet_service.cc:1467] Processing CreateTablet for tablet 421c7506808c4b1ea5d025087231529c (DEFAULT_TABLE table=test-table [id=3fed111182914a27931dadbb641984df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:21.473975 20265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421c7506808c4b1ea5d025087231529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:21.473677 20002 tablet_service.cc:1467] Processing CreateTablet for tablet 421c7506808c4b1ea5d025087231529c (DEFAULT_TABLE table=test-table [id=3fed111182914a27931dadbb641984df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:21.474121 20134 tablet_service.cc:1467] Processing CreateTablet for tablet 421c7506808c4b1ea5d025087231529c (DEFAULT_TABLE table=test-table [id=3fed111182914a27931dadbb641984df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:21.475014 20002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421c7506808c4b1ea5d025087231529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:21.475512 20134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421c7506808c4b1ea5d025087231529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:21.476452 20067 heartbeater.cc:503] Master 127.5.78.62:42313 was elected leader, sending a full tablet report...
I20260926 20:49:21.488408 20346 tablet_bootstrap.cc:492] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Bootstrap starting.
I20260926 20:49:21.490828 20348 tablet_bootstrap.cc:492] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Bootstrap starting.
I20260926 20:49:21.491951 20347 tablet_bootstrap.cc:492] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Bootstrap starting.
I20260926 20:49:21.493762 20346 tablet_bootstrap.cc:654] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:21.494145 20348 tablet_bootstrap.cc:654] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:21.495214 20346 log.cc:824] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:21.495321 20348 log.cc:824] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:21.495476 20347 tablet_bootstrap.cc:654] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:21.496825 20347 log.cc:824] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:21.498548 20346 tablet_bootstrap.cc:492] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: No bootstrap required, opened a new log
I20260926 20:49:21.498570 20348 tablet_bootstrap.cc:492] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: No bootstrap required, opened a new log
I20260926 20:49:21.498824 20346 ts_tablet_manager.cc:1397] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260926 20:49:21.498844 20348 ts_tablet_manager.cc:1397] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260926 20:49:21.499873 20347 tablet_bootstrap.cc:492] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: No bootstrap required, opened a new log
I20260926 20:49:21.500139 20347 ts_tablet_manager.cc:1397] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 20:49:21.506820 20346 raft_consensus.cc:348] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.507223 20346 raft_consensus.cc:374] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:21.507385 20346 raft_consensus.cc:729] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f4277b6cfc48938da1a5e77f566889, State: Initialized, Role: FOLLOWER
I20260926 20:49:21.508059 20346 consensus_queue.cc:260] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.510003 20346 ts_tablet_manager.cc:1428] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260926 20:49:21.509748 20348 raft_consensus.cc:348] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.510306 20348 raft_consensus.cc:374] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:21.510491 20348 raft_consensus.cc:729] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984d37fcef264aac926f0d5847980591, State: Initialized, Role: FOLLOWER
I20260926 20:49:21.510952 20347 raft_consensus.cc:348] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.511180 20348 consensus_queue.cc:260] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.511418 20347 raft_consensus.cc:374] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:21.511551 20347 raft_consensus.cc:729] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5d9e1d41e4f14882309233f4a9267, State: Initialized, Role: FOLLOWER
I20260926 20:49:21.512267 20347 consensus_queue.cc:260] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.513602 20352 raft_consensus.cc:482] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:49:21.513955 20352 raft_consensus.cc:504] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.514845 20348 ts_tablet_manager.cc:1428] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260926 20:49:21.515398 20199 heartbeater.cc:503] Master 127.5.78.62:42313 was elected leader, sending a full tablet report...
I20260926 20:49:21.515836 20347 ts_tablet_manager.cc:1428] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260926 20:49:21.516556 20330 heartbeater.cc:503] Master 127.5.78.62:42313 was elected leader, sending a full tablet report...
I20260926 20:49:21.516562 20352 leader_election.cc:288] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3:38807), 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499)
I20260926 20:49:21.520135 20354 raft_consensus.cc:482] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:49:21.522678 20354 raft_consensus.cc:504] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.525108 20354 leader_election.cc:288] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437), 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499)
I20260926 20:49:21.526613 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.517311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:46847 (local address 127.5.78.3:38807)
0926 20:49:21.517530 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.517536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.518143 (+ 607us) server_negotiation.cc:407] Connection header received
0926 20:49:21.518368 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.518373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.518477 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.518558 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.520388 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.521469 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.523993 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.525312 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.525438 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.525546 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.525983 (+ 437us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.526187 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 20:49:21.527619 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.517027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38807 (local address 127.5.78.1:46847)
0926 20:49:21.517996 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.518015 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.518184 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.518792 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.518796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.518823 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.519692 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.519709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.521603 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.521608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.523829 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.523848 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.525532 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.525542 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.525720 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.525766 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.526911 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.527135 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":1134,"threads_started":1}
I20260926 20:49:21.528781 20285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421c7506808c4b1ea5d025087231529c" candidate_uuid: "57f4277b6cfc48938da1a5e77f566889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe5d9e1d41e4f14882309233f4a9267" is_pre_election: true
I20260926 20:49:21.529320 20285 raft_consensus.cc:2454] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57f4277b6cfc48938da1a5e77f566889 in term 0.
I20260926 20:49:21.530119 19955 leader_election.cc:302] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5d9e1d41e4f14882309233f4a9267, 57f4277b6cfc48938da1a5e77f566889; no voters:
I20260926 20:49:21.530588 20352 raft_consensus.cc:2781] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:49:21.530737 20352 raft_consensus.cc:482] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:49:21.530840 20352 raft_consensus.cc:3037] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:21.531944 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.519125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:49101 (local address 127.5.78.2:37499)
0926 20:49:21.519307 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.519313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.521602 (+ 2289us) server_negotiation.cc:407] Connection header received
0926 20:49:21.521842 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.521847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.521936 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.522031 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.522672 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.523432 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.529328 (+ 5896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.530688 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.530793 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.530876 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.531507 (+ 631us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.531648 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260926 20:49:21.532017 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.518616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37499 (local address 127.5.78.1:49101)
0926 20:49:21.521509 (+ 2893us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.521535 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.521697 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.522053 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.522056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.522075 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.522514 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.522525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.523568 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.523573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.529116 (+ 5543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.529127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.531182 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.531187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.531315 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.531351 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.531478 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.531648 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2763,"thread_start_us":2579,"threads_started":1}
I20260926 20:49:21.533339 20154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421c7506808c4b1ea5d025087231529c" candidate_uuid: "57f4277b6cfc48938da1a5e77f566889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d37fcef264aac926f0d5847980591" is_pre_election: true
I20260926 20:49:21.533938 20154 raft_consensus.cc:2454] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57f4277b6cfc48938da1a5e77f566889 in term 0.
I20260926 20:49:21.534443 20352 raft_consensus.cc:504] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.535295 20352 leader_election.cc:288] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [CANDIDATE]: Term 1 election: Requested vote from peers 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3:38807), 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499)
I20260926 20:49:21.535398 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.526804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:56895 (local address 127.5.78.1:39437)
0926 20:49:21.527020 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.527028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.527046 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:49:21.527096 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.527099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.527148 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.527251 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.527974 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.529116 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.532664 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.534048 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.534192 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.534297 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.534966 (+ 669us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.535116 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:49:21.535876 20285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421c7506808c4b1ea5d025087231529c" candidate_uuid: "57f4277b6cfc48938da1a5e77f566889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe5d9e1d41e4f14882309233f4a9267"
I20260926 20:49:21.536125 20285 raft_consensus.cc:3037] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:21.536211 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.525852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:39437 (local address 127.5.78.3:56895)
0926 20:49:21.526684 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.526824 (+ 140us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.526939 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.527265 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.527269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.527301 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.527657 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.527673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.529920 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.529926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.532475 (+ 2549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.532491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.534481 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.534489 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.534753 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.534788 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.535571 (+ 783us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.535734 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":36,"thread_start_us":1483,"threads_started":1}
I20260926 20:49:21.537613 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.524843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:37499 (local address 127.5.78.3:45213)
0926 20:49:21.526257 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:21.526284 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:21.526434 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:21.528501 (+ 2067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:21.528506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:21.528530 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:21.528793 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.528808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.533138 (+ 4330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.533145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.535078 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:21.535092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.536772 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.536778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:21.537001 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.537050 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 20:49:21.537148 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 20:49:21.537301 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1276,"mutex_wait_us":41}
I20260926 20:49:21.537835 20357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.525653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:45213 (local address 127.5.78.2:37499)
0926 20:49:21.528183 (+ 2530us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.528187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.528205 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:49:21.528253 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.528256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.528341 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.528956 (+ 615us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:21.529239 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.532954 (+ 3715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.535307 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.536592 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.536745 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.536873 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.537342 (+ 469us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.537503 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2370,"thread_start_us":2099,"threads_started":1}
I20260926 20:49:21.538722 20154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421c7506808c4b1ea5d025087231529c" candidate_uuid: "57f4277b6cfc48938da1a5e77f566889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d37fcef264aac926f0d5847980591"
I20260926 20:49:21.538676 20022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421c7506808c4b1ea5d025087231529c" candidate_uuid: "1fe5d9e1d41e4f14882309233f4a9267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f4277b6cfc48938da1a5e77f566889" is_pre_election: true
I20260926 20:49:21.538698 20153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421c7506808c4b1ea5d025087231529c" candidate_uuid: "1fe5d9e1d41e4f14882309233f4a9267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d37fcef264aac926f0d5847980591" is_pre_election: true
I20260926 20:49:21.539012 20154 raft_consensus.cc:3037] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:21.539208 20022 raft_consensus.cc:2379] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fe5d9e1d41e4f14882309233f4a9267 in current term 1: Already voted for candidate 57f4277b6cfc48938da1a5e77f566889 in this term.
I20260926 20:49:21.539445 20285 raft_consensus.cc:2454] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57f4277b6cfc48938da1a5e77f566889 in term 1.
I20260926 20:49:21.539920 19955 leader_election.cc:302] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5d9e1d41e4f14882309233f4a9267, 57f4277b6cfc48938da1a5e77f566889; no voters:
I20260926 20:49:21.540236 20352 raft_consensus.cc:2781] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:21.540329 20219 leader_election.cc:302] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fe5d9e1d41e4f14882309233f4a9267; no voters: 57f4277b6cfc48938da1a5e77f566889, 984d37fcef264aac926f0d5847980591
I20260926 20:49:21.540818 20352 raft_consensus.cc:686] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 1 LEADER]: Becoming Leader. State: Replica: 57f4277b6cfc48938da1a5e77f566889, State: Running, Role: LEADER
I20260926 20:49:21.541055 20354 raft_consensus.cc:2726] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 20:49:21.541496 20352 consensus_queue.cc:237] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } }
I20260926 20:49:21.542279 20154 raft_consensus.cc:2454] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57f4277b6cfc48938da1a5e77f566889 in term 1.
I20260926 20:49:21.546357 19881 catalog_manager.cc:5495] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1). New cstate: current_term: 1 leader_uuid: "57f4277b6cfc48938da1a5e77f566889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f4277b6cfc48938da1a5e77f566889" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 39437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 } health_report { overall_health: UNKNOWN } } }
I20260926 20:49:21.576861 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.570552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.5.78.1:39437)
0926 20:49:21.570747 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.570755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.570796 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:49:21.571075 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.571079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.571179 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.571350 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:49:21.571747 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.572495 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.574101 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.574628 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.574769 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.575306 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:49:21.576183 (+ 877us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:49:21.576370 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.576533 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.576628 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:49:21.586066 20154 raft_consensus.cc:1261] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Refusing update from remote peer 57f4277b6cfc48938da1a5e77f566889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:21.586120 20285 raft_consensus.cc:1261] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Refusing update from remote peer 57f4277b6cfc48938da1a5e77f566889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:21.587132 20359 consensus_queue.cc:1035] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984d37fcef264aac926f0d5847980591" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 37499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:21.587570 20352 consensus_queue.cc:1035] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe5d9e1d41e4f14882309233f4a9267" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:21.616345 19881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:21.616684 19881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:21.620697 19881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Sending DeleteTablet for 3 replicas of tablet 421c7506808c4b1ea5d025087231529c
I20260926 20:49:21.622094 20265 tablet_service.cc:1514] Processing DeleteTablet for tablet 421c7506808c4b1ea5d025087231529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:21 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:41072
I20260926 20:49:21.622136 20002 tablet_service.cc:1514] Processing DeleteTablet for tablet 421c7506808c4b1ea5d025087231529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:21 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:42162
I20260926 20:49:21.623170 20373 tablet_replica.cc:330] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: stopping tablet replica
I20260926 20:49:21.623232 20134 tablet_service.cc:1514] Processing DeleteTablet for tablet 421c7506808c4b1ea5d025087231529c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:21 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:59904
I20260926 20:49:21.623335 20374 tablet_replica.cc:330] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: stopping tablet replica
I20260926 20:49:21.623761 20374 raft_consensus.cc:2229] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:49:21.623726 20373 raft_consensus.cc:2229] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:21.624372 20373 raft_consensus.cc:2258] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:21.624480 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19919
I20260926 20:49:21.624531 20374 raft_consensus.cc:2258] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:21.625592 20375 tablet_replica.cc:330] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: stopping tablet replica
I20260926 20:49:21.626078 20375 raft_consensus.cc:2229] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:21.626677 20375 raft_consensus.cc:2258] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:21.627589 20373 ts_tablet_manager.cc:1905] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:21.627833 20374 ts_tablet_manager.cc:1905] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:21.629835 20375 ts_tablet_manager.cc:1905] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:21.635342 20373 ts_tablet_manager.cc:1918] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:21.635612 20373 log.cc:1196] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/421c7506808c4b1ea5d025087231529c
I20260926 20:49:21.636279 20373 ts_tablet_manager.cc:1939] T 421c7506808c4b1ea5d025087231529c P 1fe5d9e1d41e4f14882309233f4a9267: Deleting consensus metadata
I20260926 20:49:21.637163 20375 ts_tablet_manager.cc:1918] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:21.637310 20374 ts_tablet_manager.cc:1918] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:21.637422 20375 log.cc:1196] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/421c7506808c4b1ea5d025087231529c
I20260926 20:49:21.637614 20374 log.cc:1196] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/421c7506808c4b1ea5d025087231529c
I20260926 20:49:21.638139 20375 ts_tablet_manager.cc:1939] T 421c7506808c4b1ea5d025087231529c P 984d37fcef264aac926f0d5847980591: Deleting consensus metadata
I20260926 20:49:21.638352 20374 ts_tablet_manager.cc:1939] T 421c7506808c4b1ea5d025087231529c P 57f4277b6cfc48938da1a5e77f566889: Deleting consensus metadata
I20260926 20:49:21.638527 19866 catalog_manager.cc:4841] TS 1fe5d9e1d41e4f14882309233f4a9267 (127.5.78.3:38807): tablet 421c7506808c4b1ea5d025087231529c (table test-table [id=3fed111182914a27931dadbb641984df]) successfully deleted
I20260926 20:49:21.640192 19866 catalog_manager.cc:4841] TS 984d37fcef264aac926f0d5847980591 (127.5.78.2:37499): tablet 421c7506808c4b1ea5d025087231529c (table test-table [id=3fed111182914a27931dadbb641984df]) successfully deleted
I20260926 20:49:21.640636 19867 catalog_manager.cc:4841] TS 57f4277b6cfc48938da1a5e77f566889 (127.5.78.1:39437): tablet 421c7506808c4b1ea5d025087231529c (table test-table [id=3fed111182914a27931dadbb641984df]) successfully deleted
I20260926 20:49:21.644884 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.625661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.5.78.1:39437)
0926 20:49:21.625853 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.625859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.625883 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:49:21.626014 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.626020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.626111 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.626232 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:21.628224 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.629322 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.631437 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.632220 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.632397 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.636653 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:21.636674 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:21.636695 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:21.636753 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:21.641323 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.642374 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.642382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.642385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.642612 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.643142 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.643147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.643149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.643565 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:21.643833 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.644286 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.644480 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:49:21.645582 20062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:21.743623 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19919
I20260926 20:49:21.762629 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 20071
I20260926 20:49:21.777014 20357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.763458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.5.78.2:37499)
0926 20:49:21.763667 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.763672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.763692 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:49:21.763904 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.763909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.763990 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.764084 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:21.765449 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.766265 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.767951 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.768354 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.768460 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.771138 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:21.771153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:21.771168 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:21.771210 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:21.774105 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.774878 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.774886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.774889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.775031 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.775431 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.775435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.775437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.775828 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:21.776053 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.776413 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.776643 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:49:21.777697 20194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:21.888510 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 20071
I20260926 20:49:21.905848 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 20202
I20260926 20:49:21.920241 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:21.906624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.5.78.3:38807)
0926 20:49:21.906864 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:21.906871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:21.906977 (+ 106us) server_negotiation.cc:407] Connection header received
0926 20:49:21.907199 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:21.907204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:21.907285 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:21.907396 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:21.908708 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.909444 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.910877 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:21.911432 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:21.911577 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:21.914158 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:21.914180 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:21.914196 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:21.914237 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:21.917153 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.917938 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.917948 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.917952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.918107 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:21.918577 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:21.918583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:21.918586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:21.918983 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:21.919289 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:21.919654 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 20:49:21.919898 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:49:21.921073 20325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:22.017890 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 20202
I20260926 20:49:22.035530 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 19849
I20260926 20:49:22.036755 19912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:22.151187 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 19849
I20260926 20:49:22.173498 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42313
--webserver_interface=127.5.78.62
--webserver_port=40131
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:22.519168 20380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:22.519469 20380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:22.519656 20380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:22.526852 20380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:22.526960 20380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:22.527071 20380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:22.527182 20380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:22.527252 20380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:22.537268 20380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=40131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:22.538786 20380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:22.540318 20380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:22.553139 20385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:22.553226 20386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:22.553211 20388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:22.554242 20380 server_base.cc:1034] running on GCE node
I20260926 20:49:22.555198 20380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:22.556888 20380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:22.558133 20380 hybrid_clock.cc:648] HybridClock initialized: now 1790455762558076 us; error 48 us; skew 500 ppm
I20260926 20:49:22.558615 20380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:22 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455762, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:22.562008 20380 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:22.563925 20380 webserver.cc:460] Webserver started at http://127.5.78.62:40131/ using document root <none> and password file <none>
I20260926 20:49:22.564679 20380 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:22.564803 20380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:22.570895 20380 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:49:22.574512 20395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:22.576306 20380 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:22.576553 20380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "55feb692c1004dbfa2c14a5ac3fe8b69"
format_stamp: "Formatted at 2026-09-26 20:49:16 on dist-test-slave-z7j8"
server_key: "a2f9fa26c0e75f74079ce5146388c06a"
server_key_iv: "663d15ed6ba392d8e9726b4a6632aa9f"
server_key_version: "encryptionkey@0"
I20260926 20:49:22.577672 20380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:22.628971 20380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:22.633360 20380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:22.633689 20380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:22.653046 20380 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:42313
I20260926 20:49:22.653083 20446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:42313 every 8 connection(s)
I20260926 20:49:22.654647 20380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:22.659061 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20380
I20260926 20:49:22.660059 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:39437
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:42313
--webserver_port=42177
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:49:22.663280 20447 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:49:22.668701 20447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap starting.
I20260926 20:49:22.689476 20447 log.cc:824] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:22.705765 20447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:49:22.706339 20447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69: Bootstrap complete.
I20260926 20:49:22.716150 20447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:22.717123 20447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55feb692c1004dbfa2c14a5ac3fe8b69, State: Initialized, Role: FOLLOWER
I20260926 20:49:22.717835 20447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:22.718145 20447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:49:22.718272 20447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:49:22.718492 20447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 2 FOLLOWER]: Advancing to term 3
I20260926 20:49:22.722477 20447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:22.723006 20447 leader_election.cc:302] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55feb692c1004dbfa2c14a5ac3fe8b69; no voters:
I20260926 20:49:22.723686 20447 leader_election.cc:288] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 20:49:22.723839 20451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 20:49:22.725018 20451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [term 3 LEADER]: Becoming Leader. State: Replica: 55feb692c1004dbfa2c14a5ac3fe8b69, State: Running, Role: LEADER
I20260926 20:49:22.725636 20451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } }
I20260926 20:49:22.725979 20447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:22.728359 20453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55feb692c1004dbfa2c14a5ac3fe8b69. Latest consensus state: current_term: 3 leader_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } } }
I20260926 20:49:22.728744 20453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:22.728494 20452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55feb692c1004dbfa2c14a5ac3fe8b69" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 42313 } } }
I20260926 20:49:22.728899 20452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55feb692c1004dbfa2c14a5ac3fe8b69 [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:22.739915 20458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:22.751959 20458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:49:22.753939 20458 catalog_manager.cc:1259] Loaded cluster ID: 85aaf1fe593947aa9d6db44308190fc6
I20260926 20:49:22.754142 20458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 20:49:22.756870 20458 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2e42d019ad google::LogMessage::Fail() at ??:0
@ 0x7f2e42d0488a google::LogMessage::SendToLog() at ??:0
@ 0x7f2e42d014de google::LogMessage::Flush() at ??:0
@ 0x7f2e42d02a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2e58738512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2e58735976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2e587bd610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2e587bd439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2e4594d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2e4595091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2e45950799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2e4592beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2e48c056db start_thread at ??:0
@ 0x7f2e41b9b61f clone at ??:0
W20260926 20:49:23.082250 20449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:23.082650 20449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:23.082757 20449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:23.090762 20449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:23.090885 20449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:23.091001 20449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:49:23.101610 20449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:39437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=42177
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:23.103019 20449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:23.104408 20449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:23.118703 20488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:23.118778 20489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:23.118737 20491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:23.119604 20449 server_base.cc:1034] running on GCE node
I20260926 20:49:23.120258 20449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:23.121311 20449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:23.122574 20449 hybrid_clock.cc:648] HybridClock initialized: now 1790455763122551 us; error 49 us; skew 500 ppm
I20260926 20:49:23.122953 20449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:23 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455763, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:23.126367 20449 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:49:23.128160 20449 webserver.cc:460] Webserver started at http://127.5.78.1:42177/ using document root <none> and password file <none>
I20260926 20:49:23.128895 20449 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:23.129050 20449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:23.134693 20449 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:49:23.139745 20498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:23.141386 20449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:49:23.141616 20449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "57f4277b6cfc48938da1a5e77f566889"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "42e573978bdaf0614b6dc0640e74a5f2"
server_key_iv: "3fa0094ac7f7d4d995822f85bbe3c7e9"
server_key_version: "encryptionkey@0"
I20260926 20:49:23.142773 20449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:23.155455 20449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:23.159976 20449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:23.160284 20449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:23.161372 20449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:23.162925 20449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:23.163064 20449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:23.163198 20449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:23.163286 20449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:23.198397 20449 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:39437
I20260926 20:49:23.198607 20610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:39437 every 8 connection(s)
I20260926 20:49:23.199883 20449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:49:23.205078 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:23.203100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:49497)
0926 20:49:23.203899 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:23.204118 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":365,"threads_started":1}
W20260926 20:49:23.205801 20611 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:42313 (0 consecutive failures): Network error: Failed to ping master at 127.5.78.62:42313: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
I20260926 20:49:23.206724 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:23.206330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:53737)
0926 20:49:23.206538 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:23.206610 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 20:49:23.207736 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:23.207361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:49809)
0926 20:49:23.207538 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:23.207625 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 20:49:23.207971 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20449
W20260926 20:49:23.208055 20611 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:49:23.208643 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:37499
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:42313
--webserver_port=36713
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:23.553095 20614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:23.553453 20614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:23.553633 20614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:23.561324 20614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:23.561501 20614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:23.561681 20614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:49:23.572270 20614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:37499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=36713
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:23.573722 20614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:23.575237 20614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:23.589161 20622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:23.589148 20620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:23.589113 20619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:23.590260 20614 server_base.cc:1034] running on GCE node
I20260926 20:49:23.590952 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:23.591939 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:23.593176 20614 hybrid_clock.cc:648] HybridClock initialized: now 1790455763593114 us; error 47 us; skew 500 ppm
I20260926 20:49:23.593492 20614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:23 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455763, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:23.596866 20614 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:49:23.598548 20614 webserver.cc:460] Webserver started at http://127.5.78.2:36713/ using document root <none> and password file <none>
I20260926 20:49:23.599357 20614 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:23.599509 20614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:23.605818 20614 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:49:23.608703 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:23.610174 20614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:49:23.610360 20614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "984d37fcef264aac926f0d5847980591"
format_stamp: "Formatted at 2026-09-26 20:49:17 on dist-test-slave-z7j8"
server_key: "f6156041503d28891b3e279c1ae89def"
server_key_iv: "77bb9ce09f95abfc2eb95dae20376fb0"
server_key_version: "encryptionkey@0"
I20260926 20:49:23.611467 20614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:23.629499 20614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:23.634845 20614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:23.635190 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:23.636415 20614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:23.637910 20614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:23.638051 20614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:23.638203 20614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:23.638299 20614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:23.672221 20614 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:37499
I20260926 20:49:23.672286 20741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:37499 every 8 connection(s)
I20260926 20:49:23.673753 20614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:49:23.678750 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:23.676776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:49815)
0926 20:49:23.677551 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:23.677750 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":278,"threads_started":1}
W20260926 20:49:23.679468 20742 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:42313 (0 consecutive failures): Network error: Failed to ping master at 127.5.78.62:42313: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
I20260926 20:49:23.680353 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:23.679998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:47955)
0926 20:49:23.680159 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:23.680246 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 20:49:23.681190 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:23.680898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:56853)
0926 20:49:23.681017 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:23.681087 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 20:49:23.681504 20742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:49:23.683346 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20614
I20260926 20:49:23.684007 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:38807
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:42313
--webserver_port=34487
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:24.020452 20745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:24.020726 20745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:24.020893 20745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:24.028272 20745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:24.028388 20745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:24.028565 20745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:49:24.038439 20745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:38807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=34487
--tserver_master_addrs=127.5.78.62:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:24.039759 20745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:24.041100 20745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:24.054757 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:24.054859 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:24.054859 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:24.056056 20745 server_base.cc:1034] running on GCE node
I20260926 20:49:24.056845 20745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:24.057924 20745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:24.059206 20745 hybrid_clock.cc:648] HybridClock initialized: now 1790455764059118 us; error 54 us; skew 500 ppm
I20260926 20:49:24.059576 20745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:24 dist-test-slave-z7j8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455764, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:24.062999 20745 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:49:24.064544 20745 webserver.cc:460] Webserver started at http://127.5.78.3:34487/ using document root <none> and password file <none>
I20260926 20:49:24.065269 20745 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:24.065415 20745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:24.070966 20745 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:49:24.073880 20760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:24.075475 20745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:49:24.075680 20745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "1fe5d9e1d41e4f14882309233f4a9267"
format_stamp: "Formatted at 2026-09-26 20:49:18 on dist-test-slave-z7j8"
server_key: "f147804131860b8237db8c9364587019"
server_key_iv: "8260838b66048723b985e8f18db62901"
server_key_version: "encryptionkey@0"
I20260926 20:49:24.076759 20745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:24.103510 20745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:24.108492 20745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:24.108790 20745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:24.109843 20745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:24.111222 20745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:24.111346 20745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:24.111490 20745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:24.111579 20745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:24.145396 20745 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38807
I20260926 20:49:24.145515 20872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38807 every 8 connection(s)
I20260926 20:49:24.147069 20745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:49:24.148105 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20745
I20260926 20:49:24.152460 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:24.150634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:40119)
0926 20:49:24.151343 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:24.151550 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":268,"threads_started":1}
W20260926 20:49:24.153174 20873 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:42313 (0 consecutive failures): Network error: Failed to ping master at 127.5.78.62:42313: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
I20260926 20:49:24.154078 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:24.153732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:42789)
0926 20:49:24.153907 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:24.153972 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 20:49:24.154955 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:24.154622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:52401)
0926 20:49:24.154760 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:24.154841 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 20:49:24.155189 20873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:49:24.209417 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:24.208612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:48029)
0926 20:49:24.209067 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:24.209172 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":221,"threads_started":1}
I20260926 20:49:24.683035 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:24.682162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:41901)
0926 20:49:24.682705 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:24.682798 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":300,"threads_started":1}
I20260926 20:49:25.156997 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:25.155957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:55965)
0926 20:49:25.156596 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:25.156693 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":314,"threads_started":1}
I20260926 20:49:25.211277 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:25.210526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:60485)
0926 20:49:25.210985 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:25.211069 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":219,"threads_started":1}
I20260926 20:49:25.684872 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:25.684093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:59843)
0926 20:49:25.684607 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:25.684698 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
I20260926 20:49:26.159094 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:26.158090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:38035)
0926 20:49:26.158699 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:26.158861 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":337,"threads_started":1}
I20260926 20:49:26.213048 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:26.212305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:47731)
0926 20:49:26.212790 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:26.212878 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":257,"threads_started":1}
I20260926 20:49:26.686861 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:26.685930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:45889)
0926 20:49:26.686533 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:26.686640 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":292,"threads_started":1}
I20260926 20:49:27.160688 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:27.160004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:49933)
0926 20:49:27.160434 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:27.160511 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
I20260926 20:49:27.214965 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:27.214054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:57997)
0926 20:49:27.214602 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:27.214760 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":224,"threads_started":1}
I20260926 20:49:27.688736 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:27.687909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:54363)
0926 20:49:27.688432 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:27.688524 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":249,"threads_started":1}
I20260926 20:49:28.162565 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:28.161755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:37399)
0926 20:49:28.162239 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:28.162324 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":248,"threads_started":1}
I20260926 20:49:28.216797 20889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:28.215996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:50141)
0926 20:49:28.216495 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:28.216584 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":255,"threads_started":1}
I20260926 20:49:28.690496 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:28.689688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:53715)
0926 20:49:28.690163 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:28.690267 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":242,"threads_started":1}
I20260926 20:49:29.164178 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:29.163446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:47755)
0926 20:49:29.163922 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:29.164003 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":237,"threads_started":1}
I20260926 20:49:29.218782 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:29.217802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:56111)
0926 20:49:29.218366 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:29.218522 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":269,"threads_started":1}
I20260926 20:49:29.692517 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:29.691610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:35707)
0926 20:49:29.692093 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:29.692260 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":241,"threads_started":1}
I20260926 20:49:30.166010 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:30.165160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:58063)
0926 20:49:30.165719 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:30.165837 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":275,"threads_started":1}
I20260926 20:49:30.220639 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:30.219725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:53381)
0926 20:49:30.220276 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:30.220398 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":246,"threads_started":1}
I20260926 20:49:30.694478 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:30.693602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:59363)
0926 20:49:30.694141 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:30.694229 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":242,"threads_started":1}
I20260926 20:49:31.168026 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:31.167089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:36231)
0926 20:49:31.167725 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:31.167821 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":316,"threads_started":1}
I20260926 20:49:31.222573 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:31.221711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:49503)
0926 20:49:31.222234 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:31.222326 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":279,"threads_started":1}
I20260926 20:49:31.696331 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:31.695562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:35889)
0926 20:49:31.696057 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:31.696161 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260926 20:49:32.169734 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:32.169003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:32877)
0926 20:49:32.169457 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:32.169542 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":234,"threads_started":1}
I20260926 20:49:32.224476 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:32.223577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:39485)
0926 20:49:32.224201 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:32.224294 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":237,"threads_started":1}
I20260926 20:49:32.698096 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:32.697319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:43923)
0926 20:49:32.697825 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:32.697916 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":258,"threads_started":1}
I20260926 20:49:33.171409 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:33.170693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:51675)
0926 20:49:33.171149 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:33.171233 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":227,"threads_started":1}
I20260926 20:49:33.226583 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:33.225608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:35715)
0926 20:49:33.226205 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:33.226299 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":353,"threads_started":1}
I20260926 20:49:33.700026 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:33.699285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:41043)
0926 20:49:33.699758 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:33.699853 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":244,"threads_started":1}
I20260926 20:49:34.173501 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:34.172580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:54831)
0926 20:49:34.173173 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:34.173283 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":285,"threads_started":1}
I20260926 20:49:34.228608 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:34.227697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:57791)
0926 20:49:34.228188 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:34.228358 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":254,"threads_started":1}
I20260926 20:49:34.701895 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:34.701148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:56925)
0926 20:49:34.701616 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:34.701716 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
I20260926 20:49:35.175581 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:35.174701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:32925)
0926 20:49:35.175262 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:35.175368 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":331,"threads_started":1}
I20260926 20:49:35.230399 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:35.229661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:59761)
0926 20:49:35.230141 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:35.230232 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":227,"threads_started":1}
I20260926 20:49:35.703752 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:35.702966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:48525)
0926 20:49:35.703463 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:35.703572 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":253,"threads_started":1}
I20260926 20:49:36.177591 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:36.176699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:35291)
0926 20:49:36.177278 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:36.177382 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":275,"threads_started":1}
I20260926 20:49:36.232370 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:36.231545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:47203)
0926 20:49:36.232101 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:36.232192 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":292,"threads_started":1}
I20260926 20:49:36.705822 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:36.704883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:41663)
0926 20:49:36.705516 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:36.705610 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":334,"threads_started":1}
I20260926 20:49:37.179603 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:37.178772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:48569)
0926 20:49:37.179323 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:37.179415 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":278,"threads_started":1}
I20260926 20:49:37.234149 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:37.233371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:42629)
0926 20:49:37.233864 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:37.233967 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":242,"threads_started":1}
I20260926 20:49:37.707638 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:37.706890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:44291)
0926 20:49:37.707375 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:37.707458 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":256,"threads_started":1}
I20260926 20:49:38.181372 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:38.180565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:38167)
0926 20:49:38.181017 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:38.181102 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":214,"threads_started":1}
I20260926 20:49:38.236037 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:38.235315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.1:51665)
0926 20:49:38.235764 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:38.235867 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
I20260926 20:49:38.710083 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:38.709086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.2:52861)
0926 20:49:38.709763 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:38.709868 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":332,"threads_started":1}
I20260926 20:49:39.151182 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 20449
I20260926 20:49:39.176390 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:39.152549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.5.78.1:39437)
0926 20:49:39.153260 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:39.153280 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:39.153377 (+ 97us) server_negotiation.cc:407] Connection header received
0926 20:49:39.155166 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:39.155175 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:39.155610 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:39.155964 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:39.157552 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:39.158862 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:39.160122 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:39.160901 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:39.161124 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:39.169284 (+ 8160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:39.169294 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:39.169312 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:39.169360 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:39.172711 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:39.173539 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:39.173549 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:39.173557 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:39.173748 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:39.174167 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:39.174172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:39.174176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:39.174823 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:39.175155 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:39.175647 (+ 492us) server_negotiation.cc:299] Negotiation successful
0926 20:49:39.176110 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":346,"threads_started":1}
I20260926 20:49:39.177794 20606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:39.183362 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:39.182601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:42313 (local address 127.5.78.3:49965)
0926 20:49:39.183081 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:39.183174 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":252,"threads_started":1}
I20260926 20:49:39.263290 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 20449
I20260926 20:49:39.281054 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 20614
I20260926 20:49:39.299688 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:39.282076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.5.78.2:37499)
0926 20:49:39.282704 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:39.282719 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:39.282774 (+ 55us) server_negotiation.cc:407] Connection header received
0926 20:49:39.284417 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:39.284428 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:39.284877 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:39.285194 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:39.286628 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:39.287992 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:39.289278 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:39.290011 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:39.290154 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:39.292705 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:39.292712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:39.292729 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:39.292765 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:39.296023 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:39.296817 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:39.296827 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:39.296836 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:39.297027 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:39.297441 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:39.297445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:39.297447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:39.298045 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:39.298289 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:39.298879 (+ 590us) server_negotiation.cc:299] Negotiation successful
0926 20:49:39.299353 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":394,"threads_started":1}
I20260926 20:49:39.301250 20737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:39.386726 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 20614
I20260926 20:49:39.404521 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 20745
I20260926 20:49:39.424407 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:39.405719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.5.78.3:38807)
0926 20:49:39.406471 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:39.406497 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:39.406548 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:49:39.408158 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:39.408168 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:39.408632 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:39.408952 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:39.410537 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:39.411848 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:39.413202 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:39.413954 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:39.414133 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:39.417194 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:39.417204 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:39.417223 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:39.417276 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:39.420620 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:39.421574 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:39.421583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:39.421591 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:39.421804 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:39.422233 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:39.422237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:39.422240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:39.422970 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:39.423210 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:39.423752 (+ 542us) server_negotiation.cc:299] Negotiation successful
0926 20:49:39.424102 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":328,"threads_started":1}
I20260926 20:49:39.425827 20868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:39.509311 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 20745
2026-09-26T20:49:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24470 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:49:39 dist-test-slave-z7j8 krb5kdc[20931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:49:39 dist-test-slave-z7j8 krb5kdc[20931](info): set up 2 sockets
Sep 26 20:49:39 dist-test-slave-z7j8 krb5kdc[20931](info): commencing operation
krb5kdc: starting...
W20260926 20:49:40.615079 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:49:40 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:49:40.858759 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33207
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:41.195480 20944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:41.195755 20944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:41.195911 20944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:41.203099 20944 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 20:49:41.203181 20944 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 20:49:41.203253 20944 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 20:49:41.203326 20944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:49:41.203413 20944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:41.203542 20944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:41.203594 20944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:49:41.203692 20944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:49:41.213297 20944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:41.214541 20944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:41.215842 20944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:41.227595 20949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:41.227641 20950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:41.227555 20952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:41.228569 20944 server_base.cc:1034] running on GCE node
I20260926 20:49:41.229367 20944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:41.230755 20944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:41.231985 20944 hybrid_clock.cc:648] HybridClock initialized: now 1790455781231925 us; error 46 us; skew 500 ppm
I20260926 20:49:41.232365 20944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:41 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455781, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:41.235435 20944 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:49:41.236845 20944 webserver.cc:460] Webserver started at http://127.5.78.62:41625/ using document root <none> and password file <none>
I20260926 20:49:41.237515 20944 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:41.237625 20944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:41.237952 20944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:41.240963 20944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "ed6c1a3a012e455a9019cb0648ffee8e"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "8f72886d5eb4f94cd587bdb17066fbbd"
server_key_iv: "5d8abeb948361140a80bf2900a33fbd0"
server_key_version: "encryptionkey@0"
I20260926 20:49:41.241740 20944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "ed6c1a3a012e455a9019cb0648ffee8e"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "8f72886d5eb4f94cd587bdb17066fbbd"
server_key_iv: "5d8abeb948361140a80bf2900a33fbd0"
server_key_version: "encryptionkey@0"
I20260926 20:49:41.246858 20944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:49:41.250464 20959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:41.251784 20944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:49:41.251987 20944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ed6c1a3a012e455a9019cb0648ffee8e"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "8f72886d5eb4f94cd587bdb17066fbbd"
server_key_iv: "5d8abeb948361140a80bf2900a33fbd0"
server_key_version: "encryptionkey@0"
I20260926 20:49:41.252247 20944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:41.265909 20944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:41.270056 20944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:41.270340 20944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:41.289067 20944 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33207
I20260926 20:49:41.289103 21010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33207 every 8 connection(s)
I20260926 20:49:41.290403 20944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:49:41.293759 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 20944
I20260926 20:49:41.293982 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:49:41.294269 21011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:41.294919 5432 external_mini_cluster.cc:1428] Setting key a558a247749ed366ffad979b5a4cd197
I20260926 20:49:41.308283 21011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap starting.
Sep 26 20:49:41 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:41.312065 21011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:41.313349 21011 log.cc:824] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:41.316439 21011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: No bootstrap required, opened a new log
I20260926 20:49:41.317997 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:41.297130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.5.78.62:33207)
0926 20:49:41.298055 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:41.298092 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:41.298153 (+ 61us) server_negotiation.cc:407] Connection header received
0926 20:49:41.300056 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:41.300079 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:41.300461 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:41.300849 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:41.302325 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:41.303688 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:41.305055 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:41.305764 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:41.305983 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:41.310563 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:41.310571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:41.310584 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:41.310624 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:41.313879 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:41.314689 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:41.314696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:41.314703 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:41.314916 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:41.315334 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:41.315346 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:41.315348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:41.315841 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:41.316153 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:41.316615 (+ 462us) server_negotiation.cc:299] Negotiation successful
0926 20:49:41.317051 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":336,"threads_started":1}
I20260926 20:49:41.324944 21011 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:49:41.325467 21011 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:41.325594 21011 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed6c1a3a012e455a9019cb0648ffee8e, State: Initialized, Role: FOLLOWER
I20260926 20:49:41.326299 21011 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:49:41.326627 21011 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:49:41.326823 21011 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:49:41.327037 21011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:41.329599 21011 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:49:41.330220 21011 leader_election.cc:302] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed6c1a3a012e455a9019cb0648ffee8e; no voters:
I20260926 20:49:41.330905 21011 leader_election.cc:288] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:49:41.331055 21016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:41.331815 21016 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 LEADER]: Becoming Leader. State: Replica: ed6c1a3a012e455a9019cb0648ffee8e, State: Running, Role: LEADER
I20260926 20:49:41.332412 21016 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:49:41.333329 21011 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:49:41.335253 21018 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed6c1a3a012e455a9019cb0648ffee8e. Latest consensus state: current_term: 1 leader_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } } }
I20260926 20:49:41.335486 21017 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } } }
I20260926 20:49:41.335705 21018 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:41.335758 21017 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:49:41.336862 21022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:49:41.343600 21022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:49:41.353266 21022 catalog_manager.cc:1347] Generated new cluster ID: 084db432593742b2ac4231a485d2de38
I20260926 20:49:41.353404 21022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:49:41.380103 21022 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:49:41.381505 21022 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:49:41.391251 21022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Generated new TSK 0
I20260926 20:49:41.392182 21022 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:49:41.458258 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33207
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:41.800999 21037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:41.801338 21037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:41.801558 21037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:41.809258 21037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:41.809417 21037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:41.809592 21037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:49:41.819577 21037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:41.820840 21037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:41.822150 21037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:41.835382 21042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:41.835465 21045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:41.835418 21043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:41.836109 21037 server_base.cc:1034] running on GCE node
I20260926 20:49:41.836769 21037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:41.837654 21037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:41.838878 21037 hybrid_clock.cc:648] HybridClock initialized: now 1790455781838837 us; error 26 us; skew 500 ppm
I20260926 20:49:41.839246 21037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:41 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455781, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:41.842682 21037 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:49:41.844244 21037 webserver.cc:460] Webserver started at http://127.5.78.1:35893/ using document root <none> and password file <none>
I20260926 20:49:41.844970 21037 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:41.845129 21037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:41.845453 21037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:41.848469 21037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "11ed9f30ff5b4db4b231148a0307f672"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "d6bd48b92f8fd929fd9e3d05c7d7a731"
server_key_iv: "0314068992a1f1ecb9f79c65d3024a20"
server_key_version: "encryptionkey@0"
I20260926 20:49:41.849253 21037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "11ed9f30ff5b4db4b231148a0307f672"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "d6bd48b92f8fd929fd9e3d05c7d7a731"
server_key_iv: "0314068992a1f1ecb9f79c65d3024a20"
server_key_version: "encryptionkey@0"
I20260926 20:49:41.854516 21037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:49:41.858346 21052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:41.859841 21037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:49:41.860029 21037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "11ed9f30ff5b4db4b231148a0307f672"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "d6bd48b92f8fd929fd9e3d05c7d7a731"
server_key_iv: "0314068992a1f1ecb9f79c65d3024a20"
server_key_version: "encryptionkey@0"
I20260926 20:49:41.860260 21037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:41.881371 21037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:41.885874 21037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:41.886183 21037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:41.887290 21037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:41.888666 21037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:41.888767 21037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:41.888916 21037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:41.888994 21037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:41.922631 21037 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:42711
I20260926 20:49:41.922719 21164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:42711 every 8 connection(s)
I20260926 20:49:41.923909 21037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:49:41.929203 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21037
I20260926 20:49:41.929426 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:49:41.930088 5432 external_mini_cluster.cc:1428] Setting key fc97629305a5f303d7b4172fedfd8d1b
Sep 26 20:49:41 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455781, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:41.946676 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:41.926876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:36391 (local address 127.5.78.62:33207)
0926 20:49:41.927373 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:41.927378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:41.928440 (+ 1062us) server_negotiation.cc:407] Connection header received
0926 20:49:41.930488 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:41.930493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:41.930585 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:41.930711 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:41.932307 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:41.933060 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:41.935760 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:41.936196 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:41.936310 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:41.940750 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:41.940755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:41.940758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:41.940792 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:41.943443 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:41.944222 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:41.944228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:41.944234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:41.944365 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:41.944828 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:41.944833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:41.944837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:41.945116 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:41.945275 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:41.946029 (+ 754us) server_negotiation.cc:299] Negotiation successful
0926 20:49:41.946282 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":255,"threads_started":1}
I20260926 20:49:41.948262 21168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:41.927488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:36391)
0926 20:49:41.928263 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:41.928308 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:41.930145 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:41.930960 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:41.930970 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:41.931470 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:41.932123 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:41.932147 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:41.933216 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:41.933225 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:41.935561 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:41.935580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:41.936447 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:41.936456 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:41.936811 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:41.937656 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:41.937698 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:41.940508 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:41.943614 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:41.943624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:41.943642 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:41.944082 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:41.944487 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:41.944492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:41.944496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:41.944707 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:41.945248 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:41.945257 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:41.945635 (+ 378us) client_negotiation.cc:769] Sending connection context
0926 20:49:41.945927 (+ 292us) client_negotiation.cc:240] Negotiation successful
0926 20:49:41.946518 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":305,"threads_started":1}
I20260926 20:49:41.951706 21165 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33207
I20260926 20:49:41.952099 21165 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:41.953366 21165 heartbeater.cc:511] Master 127.5.78.62:33207 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:49:41.956697 20976 ts_manager.cc:194] Registered new tserver with Master: 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1:42711)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:49:41.960474 20976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:36391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:49:41.994122 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33207
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:42.318972 21171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:42.319267 21171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:42.319427 21171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:42.326719 21171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:42.326826 21171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:42.327003 21171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:49:42.337031 21171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:42.338375 21171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:42.339715 21171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:42.353340 21176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:42.353266 21179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:42.353302 21177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:42.354570 21171 server_base.cc:1034] running on GCE node
I20260926 20:49:42.355402 21171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:42.356460 21171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:42.357752 21171 hybrid_clock.cc:648] HybridClock initialized: now 1790455782357670 us; error 70 us; skew 500 ppm
I20260926 20:49:42.358127 21171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:42 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:42.361433 21171 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:49:42.363049 21171 webserver.cc:460] Webserver started at http://127.5.78.2:37003/ using document root <none> and password file <none>
I20260926 20:49:42.363750 21171 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:42.363910 21171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:42.364284 21171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:42.367535 21171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "553ddf9a6bec41d7918f563d808128c3"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "49443357b90c71a01bf1e56628ba84e3"
server_key_iv: "d66a4620187812fc61ed14989a2c8fd9"
server_key_version: "encryptionkey@0"
I20260926 20:49:42.368265 21171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "553ddf9a6bec41d7918f563d808128c3"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "49443357b90c71a01bf1e56628ba84e3"
server_key_iv: "d66a4620187812fc61ed14989a2c8fd9"
server_key_version: "encryptionkey@0"
I20260926 20:49:42.373615 21171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 20:49:42.377640 21186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:42.379258 21171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:49:42.379459 21171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "553ddf9a6bec41d7918f563d808128c3"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "49443357b90c71a01bf1e56628ba84e3"
server_key_iv: "d66a4620187812fc61ed14989a2c8fd9"
server_key_version: "encryptionkey@0"
I20260926 20:49:42.379709 21171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:42.400501 21171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:42.405014 21171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:42.405326 21171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:42.406315 21171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:42.407811 21171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:42.407912 21171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:42.408067 21171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:42.408147 21171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:42.442571 21171 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:33895
I20260926 20:49:42.442638 21298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:33895 every 8 connection(s)
I20260926 20:49:42.443859 21171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:49:42.448405 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21171
I20260926 20:49:42.448633 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:49:42.449258 5432 external_mini_cluster.cc:1428] Setting key 636e197d93265b8a31dbcf4c0290aec9
Sep 26 20:49:42 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:42.467250 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:42.446545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:40741 (local address 127.5.78.62:33207)
0926 20:49:42.446766 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:42.446772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:42.448012 (+ 1240us) server_negotiation.cc:407] Connection header received
0926 20:49:42.450602 (+ 2590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:42.450608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:42.450689 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:42.450802 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:42.452460 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.453657 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.455382 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.456160 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.456312 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:42.460661 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:42.460666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:42.460670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:42.460704 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:42.463484 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:42.464591 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:42.464596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:42.464598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:42.464792 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:42.465351 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:42.465355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:42.465358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:42.465638 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:42.465758 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:42.466793 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0926 20:49:42.466977 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:49:42.468791 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:42.447133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:40741)
0926 20:49:42.447841 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:42.447882 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:42.450280 (+ 2398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:42.451010 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:42.451018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:42.451701 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:42.452276 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:42.452295 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.453819 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.453823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:42.455204 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:42.455216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.456420 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.456428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:42.456680 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:42.457446 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:42.457473 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:42.460395 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:42.463684 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:42.463697 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:42.463718 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:42.464401 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:42.464918 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:42.464926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:42.464933 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:42.465220 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:42.465784 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:42.465796 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:42.466302 (+ 506us) client_negotiation.cc:769] Sending connection context
0926 20:49:42.466657 (+ 355us) client_negotiation.cc:240] Negotiation successful
0926 20:49:42.467265 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":281,"threads_started":1}
I20260926 20:49:42.470693 21299 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33207
I20260926 20:49:42.471192 21299 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:42.472018 21299 heartbeater.cc:511] Master 127.5.78.62:33207 requested a full tablet report, sending...
I20260926 20:49:42.473673 20976 ts_manager.cc:194] Registered new tserver with Master: 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2:33895)
I20260926 20:49:42.474588 20976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:40741
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:49:42.506582 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33207
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:49:42.834100 21304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:49:42.834463 21304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:49:42.834601 21304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:49:42.841967 21304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:49:42.842074 21304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:49:42.842187 21304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:49:42.851951 21304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:49:42.853191 21304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:49:42.854482 21304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:49:42.867872 21312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:42.867926 21309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:49:42.867904 21310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:49:42.868999 21304 server_base.cc:1034] running on GCE node
I20260926 20:49:42.869541 21304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:49:42.870464 21304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:49:42.871675 21304 hybrid_clock.cc:648] HybridClock initialized: now 1790455782871634 us; error 29 us; skew 500 ppm
I20260926 20:49:42.871992 21304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:49:42 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:49:42.875160 21304 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:49:42.876492 21304 webserver.cc:460] Webserver started at http://127.5.78.3:42563/ using document root <none> and password file <none>
I20260926 20:49:42.877177 21304 fs_manager.cc:359] Metadata directory not provided
I20260926 20:49:42.877303 21304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:49:42.877625 21304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:49:42.880707 21304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "956d1d86903d4be087a9b0507b1f3954"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "64164f54c6397e50b98aed0691dbc537"
server_key_iv: "5756214e0068d53078fed9e567a28935"
server_key_version: "encryptionkey@0"
I20260926 20:49:42.881834 21304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "956d1d86903d4be087a9b0507b1f3954"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "64164f54c6397e50b98aed0691dbc537"
server_key_iv: "5756214e0068d53078fed9e567a28935"
server_key_version: "encryptionkey@0"
I20260926 20:49:42.887960 21304 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 20:49:42.891777 21319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:42.893160 21304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:49:42.893349 21304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "956d1d86903d4be087a9b0507b1f3954"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "64164f54c6397e50b98aed0691dbc537"
server_key_iv: "5756214e0068d53078fed9e567a28935"
server_key_version: "encryptionkey@0"
I20260926 20:49:42.893592 21304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:49:42.933162 21304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:49:42.938253 21304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:49:42.938612 21304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:49:42.939688 21304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:49:42.941110 21304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:49:42.941214 21304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:42.941372 21304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:49:42.941457 21304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:49:42.965643 21165 heartbeater.cc:503] Master 127.5.78.62:33207 was elected leader, sending a full tablet report...
I20260926 20:49:42.977828 21304 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:33535
I20260926 20:49:42.977969 21431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:33535 every 8 connection(s)
I20260926 20:49:42.979202 21304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:49:42.980643 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21304
I20260926 20:49:42.980825 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:49:42.981344 5432 external_mini_cluster.cc:1428] Setting key 4e3c657eec13547a93a0c72cbbf1ef1d
Sep 26 20:49:42 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455782, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:49:43.000000 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:42.981935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:44551 (local address 127.5.78.62:33207)
0926 20:49:42.982379 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:42.982384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:42.983612 (+ 1228us) server_negotiation.cc:407] Connection header received
0926 20:49:42.985528 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:42.985532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:42.985596 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:42.985693 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:42.987179 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.987972 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.989508 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.989927 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.990043 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:42.993563 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:42.993568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:42.993571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:42.993614 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:42.996606 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:42.997641 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:42.997646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:42.997648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:42.997791 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:42.998233 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:42.998237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:42.998239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:42.998619 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:42.998737 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:42.999522 (+ 785us) server_negotiation.cc:299] Negotiation successful
0926 20:49:42.999702 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":204,"threads_started":1}
I20260926 20:49:43.000881 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:42.982618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:44551)
0926 20:49:42.983447 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:42.983504 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:42.985258 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:42.985904 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:42.985912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:42.986471 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:49:42.987031 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:42.987048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.988131 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.988135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:42.989360 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:42.989372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:42.990143 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:42.990148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:42.990343 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:42.991078 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:49:42.991100 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:49:42.993321 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:49:42.996735 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:42.996745 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:42.996764 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:42.997452 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:42.997905 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:49:42.997909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:49:42.997911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:49:42.998129 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:49:42.998741 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:49:42.998747 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:49:42.999109 (+ 362us) client_negotiation.cc:769] Sending connection context
0926 20:49:42.999348 (+ 239us) client_negotiation.cc:240] Negotiation successful
0926 20:49:42.999758 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":381,"threads_started":1}
I20260926 20:49:43.002508 21432 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33207
I20260926 20:49:43.002862 21432 heartbeater.cc:464] Registering TS with master...
I20260926 20:49:43.003690 21432 heartbeater.cc:511] Master 127.5.78.62:33207 requested a full tablet report, sending...
I20260926 20:49:43.005311 20976 ts_manager.cc:194] Registered new tserver with Master: 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535)
I20260926 20:49:43.006171 20976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:44551
I20260926 20:49:43.009866 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:49:43.027947 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.014463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.5.78.62:33207)
0926 20:49:43.014637 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.014642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.015155 (+ 513us) server_negotiation.cc:407] Connection header received
0926 20:49:43.015306 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.015310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.015389 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.015478 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:43.016922 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.017649 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.019097 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.019505 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.019610 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.022334 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:43.022340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:43.022343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:43.022377 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:43.025195 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.026077 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.026081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.026084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.026208 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.026677 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.026681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.026683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.027029 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:43.027224 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.027517 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.027707 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:49:43.032047 20976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:49:43.039418 20976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:49:43.065060 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.052610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.5.78.2:33895)
0926 20:49:43.053966 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.053980 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.054947 (+ 967us) server_negotiation.cc:407] Connection header received
0926 20:49:43.055094 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.055103 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.055365 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.055563 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:43.056878 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.058348 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.061225 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.062772 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.062916 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.064358 (+ 1442us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.064445 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.064663 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":1000,"threads_started":1}
I20260926 20:49:43.065114 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.053539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:33535 (local address 127.0.0.1:57722)
0926 20:49:43.054519 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:43.054570 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:43.054751 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:43.056437 (+ 1686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:43.056448 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:43.056483 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:43.057087 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.057099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.059420 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.059426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.061892 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.061906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.063816 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.063823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.064392 (+ 569us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.064427 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:49:43.064494 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 20:49:43.064625 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"spinlock_wait_cycles":9344,"thread_start_us":910,"threads_started":1}
I20260926 20:49:43.065573 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.052599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:33895 (local address 127.0.0.1:37268)
0926 20:49:43.054520 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:43.054572 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:43.054751 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:43.055562 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:43.055569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:43.055614 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:43.056642 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.056657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.058508 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.058513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.061073 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.061084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.063286 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.063293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.063522 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.063601 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 20:49:43.063740 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 20:49:43.063984 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1608,"spinlock_wait_cycles":8576,"thread_start_us":1584,"threads_started":1}
I20260926 20:49:43.066505 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.055676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:51684)
0926 20:49:43.057319 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:43.057338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:43.057455 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:43.058019 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:43.058024 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:43.058046 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:43.058447 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.058463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.060178 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.060183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.062539 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.062554 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.065719 (+ 3165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.065725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.065897 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.065929 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:49:43.066026 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:49:43.066161 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1521,"thread_start_us":1362,"threads_started":1}
I20260926 20:49:43.066524 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.054218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.5.78.3:33535)
0926 20:49:43.055426 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.055435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.055793 (+ 358us) server_negotiation.cc:407] Connection header received
0926 20:49:43.055906 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.055915 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.056179 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.057294 (+ 1115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:43.057587 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.059218 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.062074 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.063526 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.065655 (+ 2129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.065912 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.065982 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.066151 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":857,"threads_started":1}
I20260926 20:49:43.066679 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.055346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.5.78.1:42711)
0926 20:49:43.056220 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.056232 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.057448 (+ 1216us) server_negotiation.cc:407] Connection header received
0926 20:49:43.057587 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.057594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.057822 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.058014 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:43.058597 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.060040 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.062709 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.064224 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.065170 (+ 946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.065518 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.066095 (+ 577us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.066299 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":557,"threads_started":1}
I20260926 20:49:43.070778 21367 tablet_service.cc:1467] Processing CreateTablet for tablet 56b9ad959f024a84b471e5cb102449a4 (DEFAULT_TABLE table=test-table [id=0670decd1a0146f7b18306133c15b608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:43.072391 21367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b9ad959f024a84b471e5cb102449a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:43.072055 21234 tablet_service.cc:1467] Processing CreateTablet for tablet 56b9ad959f024a84b471e5cb102449a4 (DEFAULT_TABLE table=test-table [id=0670decd1a0146f7b18306133c15b608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:43.072309 21100 tablet_service.cc:1467] Processing CreateTablet for tablet 56b9ad959f024a84b471e5cb102449a4 (DEFAULT_TABLE table=test-table [id=0670decd1a0146f7b18306133c15b608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:49:43.073352 21234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b9ad959f024a84b471e5cb102449a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:43.073559 21100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b9ad959f024a84b471e5cb102449a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:49:43.084410 21451 tablet_bootstrap.cc:492] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Bootstrap starting.
I20260926 20:49:43.087740 21451 tablet_bootstrap.cc:654] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:43.087728 21452 tablet_bootstrap.cc:492] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Bootstrap starting.
I20260926 20:49:43.089887 21451 log.cc:824] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:43.091336 21452 tablet_bootstrap.cc:654] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:43.092608 21452 log.cc:824] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:43.093364 21451 tablet_bootstrap.cc:492] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: No bootstrap required, opened a new log
I20260926 20:49:43.093650 21451 ts_tablet_manager.cc:1397] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 20:49:43.094591 21453 tablet_bootstrap.cc:492] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Bootstrap starting.
I20260926 20:49:43.095850 21452 tablet_bootstrap.cc:492] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: No bootstrap required, opened a new log
I20260926 20:49:43.096143 21452 ts_tablet_manager.cc:1397] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:49:43.097748 21453 tablet_bootstrap.cc:654] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Neither blocks nor log segments found. Creating new log.
I20260926 20:49:43.099063 21453 log.cc:824] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Log is configured to *not* fsync() on all Append() calls
I20260926 20:49:43.104744 21453 tablet_bootstrap.cc:492] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: No bootstrap required, opened a new log
I20260926 20:49:43.105031 21453 ts_tablet_manager.cc:1397] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260926 20:49:43.104969 21451 raft_consensus.cc:348] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.105427 21451 raft_consensus.cc:374] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:43.105526 21451 raft_consensus.cc:729] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d1d86903d4be087a9b0507b1f3954, State: Initialized, Role: FOLLOWER
I20260926 20:49:43.106204 21451 consensus_queue.cc:260] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.106607 21452 raft_consensus.cc:348] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.107061 21452 raft_consensus.cc:374] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:43.107173 21452 raft_consensus.cc:729] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11ed9f30ff5b4db4b231148a0307f672, State: Initialized, Role: FOLLOWER
I20260926 20:49:43.107559 21432 heartbeater.cc:503] Master 127.5.78.62:33207 was elected leader, sending a full tablet report...
I20260926 20:49:43.107846 21452 consensus_queue.cc:260] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.108104 21451 ts_tablet_manager.cc:1428] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260926 20:49:43.109858 21452 ts_tablet_manager.cc:1428] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260926 20:49:43.115345 21453 raft_consensus.cc:348] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.115844 21453 raft_consensus.cc:374] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:49:43.115962 21453 raft_consensus.cc:729] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 553ddf9a6bec41d7918f563d808128c3, State: Initialized, Role: FOLLOWER
I20260926 20:49:43.116513 21453 consensus_queue.cc:260] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.117522 21299 heartbeater.cc:503] Master 127.5.78.62:33207 was elected leader, sending a full tablet report...
I20260926 20:49:43.118183 21453 ts_tablet_manager.cc:1428] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260926 20:49:43.176369 21166 tablet.cc:2369] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:49:43.195952 21300 tablet.cc:2369] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:43.206310 21459 raft_consensus.cc:482] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:49:43.206861 21459 raft_consensus.cc:504] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.208585 21459 leader_election.cc:288] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1:42711), 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535)
I20260926 20:49:43.217692 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.209097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:60201 (local address 127.5.78.1:42711)
0926 20:49:43.209329 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.209337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.209994 (+ 657us) server_negotiation.cc:407] Connection header received
0926 20:49:43.210183 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.210189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.210281 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.210396 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:43.211289 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.212392 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.215664 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.216860 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.217008 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.217116 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.217355 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.217485 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:49:43.217934 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.209033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.5.78.2:60201)
0926 20:49:43.209830 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:43.209853 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:43.210031 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:43.210446 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:43.210451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:43.210545 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:43.211097 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.211105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.212588 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.212594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.215434 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.215449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.217073 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.217078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.217196 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.217221 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:49:43.217300 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 20:49:43.217476 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":183,"spinlock_wait_cycles":24960,"thread_start_us":456,"threads_started":1}
I20260926 20:49:43.218254 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.209090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:33535 (local address 127.5.78.2:49689)
0926 20:49:43.209932 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0926 20:49:43.209947 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:49:43.210060 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:49:43.210448 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:49:43.210451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:49:43.210477 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:49:43.211089 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.211105 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.212588 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.212594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.215437 (+ 2843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:49:43.215449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.217470 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.217475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:49:43.217656 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.217690 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:49:43.217789 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 20:49:43.217937 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":178,"thread_start_us":447,"threads_started":1}
I20260926 20:49:43.218461 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.209200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:49689 (local address 127.5.78.3:33535)
0926 20:49:43.209472 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.209481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.210054 (+ 573us) server_negotiation.cc:407] Connection header received
0926 20:49:43.210182 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.210189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.210281 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.210396 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:49:43.211289 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.212452 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.216040 (+ 3588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.217247 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.217387 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.217478 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.218070 (+ 592us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.218205 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:49:43.219316 21387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b9ad959f024a84b471e5cb102449a4" candidate_uuid: "553ddf9a6bec41d7918f563d808128c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d1d86903d4be087a9b0507b1f3954" is_pre_election: true
I20260926 20:49:43.219400 21120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b9ad959f024a84b471e5cb102449a4" candidate_uuid: "553ddf9a6bec41d7918f563d808128c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11ed9f30ff5b4db4b231148a0307f672" is_pre_election: true
I20260926 20:49:43.219856 21387 raft_consensus.cc:2454] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 553ddf9a6bec41d7918f563d808128c3 in term 0.
I20260926 20:49:43.219892 21120 raft_consensus.cc:2454] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 553ddf9a6bec41d7918f563d808128c3 in term 0.
I20260926 20:49:43.220551 21190 leader_election.cc:302] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 11ed9f30ff5b4db4b231148a0307f672, 553ddf9a6bec41d7918f563d808128c3, 956d1d86903d4be087a9b0507b1f3954; no voters:
I20260926 20:49:43.220948 21459 raft_consensus.cc:2781] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:49:43.221101 21459 raft_consensus.cc:482] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:49:43.221210 21459 raft_consensus.cc:3037] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:43.223397 21459 raft_consensus.cc:504] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
I20260926 20:49:43.223973 21459 leader_election.cc:288] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [CANDIDATE]: Term 1 election: Requested vote from peers 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1:42711), 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535)
I20260926 20:49:43.224368 21120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b9ad959f024a84b471e5cb102449a4" candidate_uuid: "553ddf9a6bec41d7918f563d808128c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11ed9f30ff5b4db4b231148a0307f672"
I20260926 20:49:43.224411 21387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b9ad959f024a84b471e5cb102449a4" candidate_uuid: "553ddf9a6bec41d7918f563d808128c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d1d86903d4be087a9b0507b1f3954"
I20260926 20:49:43.224594 21120 raft_consensus.cc:3037] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:43.224598 21387 raft_consensus.cc:3037] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:49:43.227864 21387 raft_consensus.cc:2454] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 553ddf9a6bec41d7918f563d808128c3 in term 1.
I20260926 20:49:43.227857 21120 raft_consensus.cc:2454] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 553ddf9a6bec41d7918f563d808128c3 in term 1.
I20260926 20:49:43.228310 21188 leader_election.cc:302] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11ed9f30ff5b4db4b231148a0307f672, 553ddf9a6bec41d7918f563d808128c3; no voters:
I20260926 20:49:43.228645 21459 raft_consensus.cc:2781] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:49:43.229151 21459 raft_consensus.cc:686] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 1 LEADER]: Becoming Leader. State: Replica: 553ddf9a6bec41d7918f563d808128c3, State: Running, Role: LEADER
I20260926 20:49:43.229628 21459 consensus_queue.cc:237] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } }
W20260926 20:49:43.230840 21433 tablet.cc:2369] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:49:43.234222 20974 catalog_manager.cc:5495] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "553ddf9a6bec41d7918f563d808128c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } health_report { overall_health: UNKNOWN } } }
I20260926 20:49:43.266222 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.260452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.5.78.2:33895)
0926 20:49:43.260663 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.260669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.260690 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:49:43.260813 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.260817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.260906 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.261067 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:49:43.261441 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.262247 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.263864 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.264279 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.264389 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.264995 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:49:43.265541 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:49:43.265662 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.265819 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.265892 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:49:43.274385 21120 raft_consensus.cc:1261] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Refusing update from remote peer 553ddf9a6bec41d7918f563d808128c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:43.274379 21387 raft_consensus.cc:1261] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Refusing update from remote peer 553ddf9a6bec41d7918f563d808128c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:49:43.275569 21462 consensus_queue.cc:1035] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:43.275980 21459 consensus_queue.cc:1035] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:49:43.288158 21465 mvcc.cc:205] Tried to move back new op lower bound from 7333706888282001408 to 7333706888112701440. Current Snapshot: MvccSnapshot[applied={T|T < 7333706888282001408}]
I20260926 20:49:43.309346 20975 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:43.309741 20975 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:49:43.314291 20975 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Sending DeleteTablet for 3 replicas of tablet 56b9ad959f024a84b471e5cb102449a4
I20260926 20:49:43.315763 21100 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b9ad959f024a84b471e5cb102449a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:43 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:51684
I20260926 20:49:43.315759 21234 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b9ad959f024a84b471e5cb102449a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:43 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:37268
I20260926 20:49:43.316603 21367 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b9ad959f024a84b471e5cb102449a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:49:43 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:57722
I20260926 20:49:43.317106 21476 tablet_replica.cc:330] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: stopping tablet replica
I20260926 20:49:43.317595 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21037
I20260926 20:49:43.317862 21476 raft_consensus.cc:2229] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:49:43.319674 21476 raft_consensus.cc:2258] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:43.320693 21475 tablet_replica.cc:330] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: stopping tablet replica
I20260926 20:49:43.321305 21475 raft_consensus.cc:2229] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:43.323053 21476 ts_tablet_manager.cc:1905] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:43.330224 21477 tablet_replica.cc:330] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: stopping tablet replica
I20260926 20:49:43.330741 21477 raft_consensus.cc:2229] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:49:43.331149 21475 raft_consensus.cc:2258] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:43.331245 21477 raft_consensus.cc:2258] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:49:43.334633 21477 ts_tablet_manager.cc:1905] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:49:43.336335 21475 ts_tablet_manager.cc:1905] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 20:49:43 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:49:43.342067 21477 ts_tablet_manager.cc:1918] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:43.342280 21477 log.cc:1196] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/56b9ad959f024a84b471e5cb102449a4
I20260926 20:49:43.342865 21477 ts_tablet_manager.cc:1939] T 56b9ad959f024a84b471e5cb102449a4 P 956d1d86903d4be087a9b0507b1f3954: Deleting consensus metadata
I20260926 20:49:43.344913 20963 catalog_manager.cc:4841] TS 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535): tablet 56b9ad959f024a84b471e5cb102449a4 (table test-table [id=0670decd1a0146f7b18306133c15b608]) successfully deleted
I20260926 20:49:43.345028 21476 ts_tablet_manager.cc:1918] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:43.345451 21476 log.cc:1196] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/56b9ad959f024a84b471e5cb102449a4
I20260926 20:49:43.346144 21476 ts_tablet_manager.cc:1939] T 56b9ad959f024a84b471e5cb102449a4 P 553ddf9a6bec41d7918f563d808128c3: Deleting consensus metadata
I20260926 20:49:43.348275 20961 catalog_manager.cc:4841] TS 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2:33895): tablet 56b9ad959f024a84b471e5cb102449a4 (table test-table [id=0670decd1a0146f7b18306133c15b608]) successfully deleted
I20260926 20:49:43.348848 21475 ts_tablet_manager.cc:1918] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:49:43.348924 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.318663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.5.78.1:42711)
0926 20:49:43.318914 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.318923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.321532 (+ 2609us) server_negotiation.cc:407] Connection header received
0926 20:49:43.321854 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.321860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.321981 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.322076 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:43.324335 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.325505 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.332228 (+ 6723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.332986 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.333109 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.339662 (+ 6553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:43.339681 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:43.339701 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:43.339751 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:43.344136 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.345297 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.345310 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.345313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.345521 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.346201 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.346207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.346211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.346768 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:43.347117 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.347782 (+ 665us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.348078 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 20:49:43.349085 21475 log.cc:1196] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/56b9ad959f024a84b471e5cb102449a4
I20260926 20:49:43.349627 21475 ts_tablet_manager.cc:1939] T 56b9ad959f024a84b471e5cb102449a4 P 11ed9f30ff5b4db4b231148a0307f672: Deleting consensus metadata
I20260926 20:49:43.350607 21160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:43.440595 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21037
I20260926 20:49:43.440713 20961 catalog_manager.cc:4841] TS 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1:42711): tablet 56b9ad959f024a84b471e5cb102449a4 (table test-table [id=0670decd1a0146f7b18306133c15b608]) successfully deleted
I20260926 20:49:43.464512 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21171
Sep 26 20:49:43 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:49:43.484041 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.465381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.5.78.2:33895)
0926 20:49:43.465529 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.465534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.465557 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:49:43.465774 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.465778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.465835 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.465920 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:43.467670 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.468581 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.470576 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.471048 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.471161 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.476605 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:43.476626 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:43.476647 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:43.476705 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:43.480723 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.481580 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.481591 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.481594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.481764 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.482223 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.482228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.482232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.482729 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:43.483075 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.483413 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.483647 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 20:49:43.484984 21294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:43.599864 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21171
I20260926 20:49:43.618494 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21304
Sep 26 20:49:43 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:49:43.635257 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:49:43.619205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.5.78.3:33535)
0926 20:49:43.619371 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 20:49:43.619375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:49:43.619392 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:49:43.619561 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:49:43.619565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:49:43.619630 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:49:43.619720 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:49:43.621051 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.621763 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.623979 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:49:43.624382 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:49:43.624487 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:49:43.629248 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:49:43.629265 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:49:43.629280 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:49:43.629321 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:49:43.632315 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.633171 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.633185 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.633188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.633351 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:49:43.633715 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:49:43.633718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:49:43.633720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:49:43.634079 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:49:43.634338 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 20:49:43.634837 (+ 499us) server_negotiation.cc:299] Negotiation successful
0926 20:49:43.635004 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:49:43.635931 21427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:43.724751 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21304
I20260926 20:49:43.743580 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 20944
I20260926 20:49:43.744787 21006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:49:43.855408 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 20944
I20260926 20:50:03.882292 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33207
--webserver_interface=127.5.78.62
--webserver_port=41625
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:04.231220 21483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:04.231564 21483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:04.231660 21483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:04.239284 21483 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 20:50:04.239418 21483 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 20:50:04.239459 21483 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 20:50:04.239506 21483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:50:04.239563 21483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:04.239630 21483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:04.239681 21483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:50:04.239728 21483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:50:04.250027 21483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=41625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:04.251451 21483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:04.252849 21483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:04.278841 21490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:04.278750 21492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:04.278801 21489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:04.279299 21483 server_base.cc:1034] running on GCE node
I20260926 20:50:04.280400 21483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:04.282326 21483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:04.283665 21483 hybrid_clock.cc:648] HybridClock initialized: now 1790455804283550 us; error 81 us; skew 500 ppm
I20260926 20:50:04.284164 21483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:04 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:04.293021 21483 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:50:04.295511 21483 webserver.cc:460] Webserver started at http://127.5.78.62:41625/ using document root <none> and password file <none>
I20260926 20:50:04.296495 21483 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:04.296689 21483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:04.303988 21483 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:50:04.307929 21499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:04.309829 21483 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:50:04.310096 21483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ed6c1a3a012e455a9019cb0648ffee8e"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "8f72886d5eb4f94cd587bdb17066fbbd"
server_key_iv: "5d8abeb948361140a80bf2900a33fbd0"
server_key_version: "encryptionkey@0"
I20260926 20:50:04.311476 21483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:04.327108 21483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:04.332760 21483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:04.333140 21483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:04.354056 21483 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33207
I20260926 20:50:04.354095 21550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33207 every 8 connection(s)
I20260926 20:50:04.355852 21483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:50:04.358099 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21483
I20260926 20:50:04.359002 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:42711
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:33207
--webserver_port=35893
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:50:04.363839 21551 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:50:04.367568 21551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap starting.
I20260926 20:50:04.389003 21551 log.cc:824] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:04.401443 21551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:50:04.402097 21551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap complete.
I20260926 20:50:04.412029 21551 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:04.413159 21551 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed6c1a3a012e455a9019cb0648ffee8e, State: Initialized, Role: FOLLOWER
I20260926 20:50:04.413938 21551 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:04.414328 21551 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:50:04.414646 21551 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:50:04.414928 21551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:50:04.420063 21551 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:04.420780 21551 leader_election.cc:302] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed6c1a3a012e455a9019cb0648ffee8e; no voters:
I20260926 20:50:04.421541 21551 leader_election.cc:288] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:50:04.421707 21555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:50:04.423000 21555 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 LEADER]: Becoming Leader. State: Replica: ed6c1a3a012e455a9019cb0648ffee8e, State: Running, Role: LEADER
I20260926 20:50:04.423784 21555 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:04.424360 21551 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:50:04.426792 21556 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } } }
I20260926 20:50:04.427356 21556 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:04.426792 21557 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed6c1a3a012e455a9019cb0648ffee8e. Latest consensus state: current_term: 2 leader_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } } }
I20260926 20:50:04.428045 21557 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:04.435158 21561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:50:04.444501 21561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:50:04.446305 21561 catalog_manager.cc:1259] Loaded cluster ID: 084db432593742b2ac4231a485d2de38
I20260926 20:50:04.446558 21561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:50:04.451191 21561 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:50:04.453198 21561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Loaded TSK: 0
I20260926 20:50:04.502658 21561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Generated new TSK 1
I20260926 20:50:04.503244 21561 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:50:04.767314 21553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:04.767645 21553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:04.767832 21553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:04.776067 21553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:04.776232 21553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:04.776470 21553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:50:04.787279 21553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:42711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=35893
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:04.788749 21553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:04.790205 21553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:04.806633 21579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:04.806671 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:04.806934 21578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:04.808429 21553 server_base.cc:1034] running on GCE node
I20260926 20:50:04.809098 21553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:04.810127 21553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:04.811399 21553 hybrid_clock.cc:648] HybridClock initialized: now 1790455804811349 us; error 41 us; skew 500 ppm
I20260926 20:50:04.811849 21553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:04 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:04.815285 21553 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:50:04.817201 21553 webserver.cc:460] Webserver started at http://127.5.78.1:35893/ using document root <none> and password file <none>
I20260926 20:50:04.817988 21553 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:04.818142 21553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:04.824213 21553 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:50:04.827894 21588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:04.829670 21553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:50:04.829926 21553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "11ed9f30ff5b4db4b231148a0307f672"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "d6bd48b92f8fd929fd9e3d05c7d7a731"
server_key_iv: "0314068992a1f1ecb9f79c65d3024a20"
server_key_version: "encryptionkey@0"
I20260926 20:50:04.831236 21553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:04.855865 21553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:04.860663 21553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:04.860980 21553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:04.862087 21553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:04.863646 21553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:04.863777 21553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:04.863934 21553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:04.864024 21553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:04.898942 21553 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:42711
I20260926 20:50:04.899039 21700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:42711 every 8 connection(s)
I20260926 20:50:04.900509 21553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:50:04.906793 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21553
I20260926 20:50:04.907581 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:33895
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:33207
--webserver_port=37003
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:50:04 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455804, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:04.935843 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:04.904122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:57719)
0926 20:50:04.905038 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:04.905092 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:04.907584 (+ 2492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:04.909338 (+ 1754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:04.909352 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:04.910213 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:04.911141 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:04.911174 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:04.917132 (+ 5958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:04.917137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:04.918647 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:04.918667 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:04.920533 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:04.920539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:04.920784 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:04.921599 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:04.921626 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:04.924488 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:04.930489 (+ 6001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:04.930499 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:04.930513 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:04.931078 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:04.931811 (+ 733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:04.931814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:04.931817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:04.932041 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:04.933289 (+ 1248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:04.933296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:04.933660 (+ 364us) client_negotiation.cc:769] Sending connection context
0926 20:50:04.933914 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 20:50:04.934366 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":384,"threads_started":1}
I20260926 20:50:04.936564 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:04.903757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:57719 (local address 127.5.78.62:33207)
0926 20:50:04.904393 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:04.904407 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:04.905284 (+ 877us) server_negotiation.cc:407] Connection header received
0926 20:50:04.908340 (+ 3056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:04.908356 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:04.908747 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:04.909091 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:04.911475 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:04.916859 (+ 5384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:04.918868 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:04.920213 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:04.920476 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:04.924947 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:04.924964 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:04.924994 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:04.925071 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:04.930185 (+ 5114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:04.931312 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:04.931327 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:04.931340 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:04.931659 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:04.932198 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:04.932206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:04.932212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:04.933097 (+ 885us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:04.933355 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:04.934259 (+ 904us) server_negotiation.cc:299] Negotiation successful
0926 20:50:04.934960 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":294,"threads_started":1}
I20260926 20:50:04.939595 21701 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33207
I20260926 20:50:04.939980 21701 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:04.940918 21701 heartbeater.cc:511] Master 127.5.78.62:33207 requested a full tablet report, sending...
I20260926 20:50:04.943441 21516 ts_manager.cc:194] Registered new tserver with Master: 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1:42711)
I20260926 20:50:04.944924 21516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:57719
W20260926 20:50:05.267913 21705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:05.268221 21705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:05.268378 21705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:05.276337 21705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:05.276455 21705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:05.276666 21705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:50:05.287125 21705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:33895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=37003
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:05.288516 21705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:05.289878 21705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:05.304132 21713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:05.304026 21710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:05.304045 21711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:05.304904 21705 server_base.cc:1034] running on GCE node
I20260926 20:50:05.305598 21705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:05.306756 21705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:05.307976 21705 hybrid_clock.cc:648] HybridClock initialized: now 1790455805307924 us; error 42 us; skew 500 ppm
I20260926 20:50:05.308391 21705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:05 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455805, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:05.311869 21705 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:50:05.313488 21705 webserver.cc:460] Webserver started at http://127.5.78.2:37003/ using document root <none> and password file <none>
I20260926 20:50:05.314227 21705 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:05.314391 21705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:05.320508 21705 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260926 20:50:05.323827 21720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:05.325799 21705 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:50:05.326061 21705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "553ddf9a6bec41d7918f563d808128c3"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "49443357b90c71a01bf1e56628ba84e3"
server_key_iv: "d66a4620187812fc61ed14989a2c8fd9"
server_key_version: "encryptionkey@0"
I20260926 20:50:05.327417 21705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:05.344297 21705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:05.349529 21705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:05.349889 21705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:05.351064 21705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:05.352666 21705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:05.352808 21705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:05.352962 21705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:05.353092 21705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:05.391016 21705 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:33895
I20260926 20:50:05.391116 21832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:33895 every 8 connection(s)
I20260926 20:50:05.392604 21705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:50:05.396900 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21705
I20260926 20:50:05.397578 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:33535
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:33207
--webserver_port=42563
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:50:05 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455805, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:05.414582 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.395450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:43391 (local address 127.5.78.62:33207)
0926 20:50:05.395668 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.395684 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.396943 (+ 1259us) server_negotiation.cc:407] Connection header received
0926 20:50:05.399039 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.399045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.399186 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.399307 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:05.400959 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.401702 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.403466 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.403926 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.404072 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.408014 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:05.408019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:05.408022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:05.408058 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:05.410951 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.411884 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.411889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.411891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.412019 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.412509 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.412513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.412515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.412900 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:05.413064 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.413871 (+ 807us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.414160 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:50:05.415740 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.395936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:43391)
0926 20:50:05.396744 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:05.396790 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:05.398724 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:05.399587 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:05.399596 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:05.400149 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:05.400757 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.400775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.401873 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.401879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.403302 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.403314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.404179 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.404186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.404428 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.405178 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:05.405202 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:05.407745 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:05.411132 (+ 3387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:05.411143 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:05.411163 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:05.411725 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:05.412146 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:05.412152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:05.412156 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:05.412371 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:05.413038 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:05.413047 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:05.413459 (+ 412us) client_negotiation.cc:769] Sending connection context
0926 20:50:05.413764 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 20:50:05.414403 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":309,"threads_started":1}
I20260926 20:50:05.417912 21833 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33207
I20260926 20:50:05.418296 21833 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:05.419134 21833 heartbeater.cc:511] Master 127.5.78.62:33207 requested a full tablet report, sending...
I20260926 20:50:05.420974 21516 ts_manager.cc:194] Registered new tserver with Master: 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2:33895)
I20260926 20:50:05.421829 21516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:43391
W20260926 20:50:05.742481 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:05.742841 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:05.743011 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:05.750738 21836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:05.750900 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:05.751072 21836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:50:05.761284 21836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:33535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=42563
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:05.762753 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:05.764119 21836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:05.777447 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:05.777595 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:05.777595 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:05.779014 21836 server_base.cc:1034] running on GCE node
I20260926 20:50:05.779671 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:05.780627 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:05.781855 21836 hybrid_clock.cc:648] HybridClock initialized: now 1790455805781806 us; error 34 us; skew 500 ppm
I20260926 20:50:05.782163 21836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:05 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455805, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:05.785532 21836 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:50:05.787269 21836 webserver.cc:460] Webserver started at http://127.5.78.3:42563/ using document root <none> and password file <none>
I20260926 20:50:05.788014 21836 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:05.788175 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:05.794107 21836 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:50:05.797231 21851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:05.798983 21836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:50:05.799181 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "956d1d86903d4be087a9b0507b1f3954"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "64164f54c6397e50b98aed0691dbc537"
server_key_iv: "5756214e0068d53078fed9e567a28935"
server_key_version: "encryptionkey@0"
I20260926 20:50:05.800258 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:05.818661 21836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:05.823187 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:05.823467 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:05.824530 21836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:05.826048 21836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:05.826184 21836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:05.826314 21836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:05.826403 21836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:05.865072 21836 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:33535
I20260926 20:50:05.865175 21963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:33535 every 8 connection(s)
I20260926 20:50:05.866604 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:50:05.873085 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 21836
Sep 26 20:50:05 dist-test-slave-z7j8 krb5kdc[20931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455805, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:05.890787 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.874201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.5.78.62:33207)
0926 20:50:05.874826 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.874830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.875381 (+ 551us) server_negotiation.cc:407] Connection header received
0926 20:50:05.875560 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.875566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.875643 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.875760 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:05.877792 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.878969 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.880871 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.881502 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.881645 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.884529 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:05.884534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:05.884538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:05.884571 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:05.887788 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.888565 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.888571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.888575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.888750 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.889144 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.889150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.889153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.889477 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:05.889760 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.890064 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.890300 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":336,"threads_started":1}
I20260926 20:50:05.891698 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.870075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:33363 (local address 127.5.78.62:33207)
0926 20:50:05.870300 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.870305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.871652 (+ 1347us) server_negotiation.cc:407] Connection header received
0926 20:50:05.873681 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.873689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.873780 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.873925 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:05.876451 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.877638 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.879844 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.880554 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.880723 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.884815 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:05.884820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:05.884823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:05.884871 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:05.887936 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.888751 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.888762 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.888764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.888877 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.889359 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.889366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.889374 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.889646 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:05.889810 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.890597 (+ 787us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.890826 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 20:50:05.892539 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.870619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:33363)
0926 20:50:05.871452 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:05.871502 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:05.873300 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:05.874390 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:05.874427 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:05.875520 (+ 1093us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:05.876253 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.876279 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.877825 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.877832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.879672 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.879686 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.880778 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.880783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.881132 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.882017 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:05.882046 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:05.884560 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:05.888102 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:05.888113 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:05.888133 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:05.888605 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:05.888998 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:05.889004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:05.889008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:05.889229 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:05.889791 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:05.889799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:05.890181 (+ 382us) client_negotiation.cc:769] Sending connection context
0926 20:50:05.890487 (+ 306us) client_negotiation.cc:240] Negotiation successful
0926 20:50:05.891085 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":334,"threads_started":1}
I20260926 20:50:05.893927 21964 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33207
I20260926 20:50:05.894284 21964 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:05.895102 21964 heartbeater.cc:511] Master 127.5.78.62:33207 requested a full tablet report, sending...
I20260926 20:50:05.896622 21515 ts_manager.cc:194] Registered new tserver with Master: 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535)
I20260926 20:50:05.897622 21515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:33363
I20260926 20:50:05.906121 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:50:05.912865 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.907284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.5.78.62:33207)
0926 20:50:05.907516 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.907524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.907781 (+ 257us) server_negotiation.cc:407] Connection header received
0926 20:50:05.907958 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.907962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.908101 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.908232 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:50:05.908671 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.909429 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.910893 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.911373 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.911488 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.911974 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:50:05.912231 (+ 257us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 20:50:05.912594 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55460: token expired
Metrics: {"server-negotiator.queue_time_us":91}
W20260926 20:50:05.913012 21703 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55460: token expired
W20260926 20:50:05.913462 21437 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.78.62:33207)
I20260926 20:50:05.927922 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.913960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.5.78.62:33207)
0926 20:50:05.914134 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.914138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.914157 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:50:05.914219 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.914223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.914299 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.914385 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:05.915848 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.916797 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.918373 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.918858 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.918965 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.921959 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:05.921964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:05.921968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:05.922004 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:05.924967 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.925836 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.925845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.925849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.926008 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:05.926528 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:05.926533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:05.926535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:05.926901 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:05.927116 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.927495 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.927685 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:50:05.931795 21515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:50:05.939553 21515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:50:05.949338 21701 heartbeater.cc:503] Master 127.5.78.62:33207 was elected leader, sending a full tablet report...
I20260926 20:50:05.963112 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.951823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:33895 (local address 127.0.0.1:34018)
0926 20:50:05.952763 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:05.952807 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:05.952973 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:05.954488 (+ 1515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:05.954496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:05.954526 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:05.954951 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.954963 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.957764 (+ 2801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.957773 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.960331 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.960346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.962087 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.962094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.962314 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.962354 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 20:50:05.962540 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 20:50:05.962726 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":2323,"threads_started":1}
I20260926 20:50:05.963104 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.952006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.5.78.2:33895)
0926 20:50:05.953974 (+ 1968us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.953981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.954002 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:50:05.954078 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.954083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.954265 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.954483 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:05.955176 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.957626 (+ 2450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.960505 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.961841 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.961973 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.962177 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.962606 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.962771 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1845,"thread_start_us":1711,"threads_started":1}
I20260926 20:50:05.963151 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.951956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:33535 (local address 127.0.0.1:36402)
0926 20:50:05.954441 (+ 2485us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:05.954462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:05.954693 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:05.955448 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:05.955456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:05.955493 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:05.955980 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.955995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.958336 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.958344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.960617 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.960631 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.962354 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.962361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.962544 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.962576 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:50:05.962665 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:50:05.962801 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2320,"thread_start_us":740,"threads_started":1}
I20260926 20:50:05.963829 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.952541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.5.78.3:33535)
0926 20:50:05.953554 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.953561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.954553 (+ 992us) server_negotiation.cc:407] Connection header received
0926 20:50:05.954887 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.954898 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.955136 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.955323 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:05.956217 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.958206 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.960774 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.961851 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.962020 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.962244 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.963279 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.963494 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":739,"threads_started":1}
I20260926 20:50:05.966207 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.955071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:46986)
0926 20:50:05.958210 (+ 3139us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:05.958239 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:05.958364 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:05.959001 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:05.959013 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:05.959038 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:05.959490 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.959502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.961280 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.961286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.963340 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:05.963354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.965404 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.965411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:05.965589 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.965621 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:50:05.965732 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 20:50:05.965866 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3003,"thread_start_us":2847,"threads_started":1}
I20260926 20:50:05.966248 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:05.954783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.5.78.1:42711)
0926 20:50:05.955338 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:05.955345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:05.958394 (+ 3049us) server_negotiation.cc:407] Connection header received
0926 20:50:05.958545 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:05.958555 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:05.958793 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:05.958992 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:05.959679 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.961138 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.963497 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:05.964982 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:05.965105 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:05.965315 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:05.965800 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 20:50:05.965959 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":416,"threads_started":1}
I20260926 20:50:05.967579 21768 tablet_service.cc:1467] Processing CreateTablet for tablet d6ea734c5b574c4db67ddc6b5cd6618f (DEFAULT_TABLE table=test-table [id=1c3df298c2754a80ac87002897714780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:05.967839 21899 tablet_service.cc:1467] Processing CreateTablet for tablet d6ea734c5b574c4db67ddc6b5cd6618f (DEFAULT_TABLE table=test-table [id=1c3df298c2754a80ac87002897714780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:05.969033 21899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ea734c5b574c4db67ddc6b5cd6618f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:05.970227 21768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ea734c5b574c4db67ddc6b5cd6618f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:05.969969 21636 tablet_service.cc:1467] Processing CreateTablet for tablet d6ea734c5b574c4db67ddc6b5cd6618f (DEFAULT_TABLE table=test-table [id=1c3df298c2754a80ac87002897714780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:05.971470 21636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ea734c5b574c4db67ddc6b5cd6618f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:05.984813 21980 tablet_bootstrap.cc:492] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Bootstrap starting.
I20260926 20:50:05.987012 21981 tablet_bootstrap.cc:492] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Bootstrap starting.
I20260926 20:50:05.987809 21982 tablet_bootstrap.cc:492] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Bootstrap starting.
I20260926 20:50:05.988804 21980 tablet_bootstrap.cc:654] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:05.990170 21980 log.cc:824] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:05.991637 21981 tablet_bootstrap.cc:654] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:05.993029 21981 log.cc:824] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:05.993863 21982 tablet_bootstrap.cc:654] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:05.994503 21980 tablet_bootstrap.cc:492] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: No bootstrap required, opened a new log
I20260926 20:50:05.994792 21980 ts_tablet_manager.cc:1397] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260926 20:50:05.995059 21982 log.cc:824] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:05.996165 21981 tablet_bootstrap.cc:492] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: No bootstrap required, opened a new log
I20260926 20:50:05.996412 21981 ts_tablet_manager.cc:1397] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 20:50:05.997437 21982 tablet_bootstrap.cc:492] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: No bootstrap required, opened a new log
I20260926 20:50:05.997685 21982 ts_tablet_manager.cc:1397] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 20:50:06.005625 21982 raft_consensus.cc:348] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.005699 21980 raft_consensus.cc:348] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.006027 21982 raft_consensus.cc:374] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:06.006115 21982 raft_consensus.cc:729] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 553ddf9a6bec41d7918f563d808128c3, State: Initialized, Role: FOLLOWER
I20260926 20:50:06.006204 21980 raft_consensus.cc:374] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:06.006338 21980 raft_consensus.cc:729] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11ed9f30ff5b4db4b231148a0307f672, State: Initialized, Role: FOLLOWER
I20260926 20:50:06.006842 21982 consensus_queue.cc:260] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.007005 21980 consensus_queue.cc:260] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.007071 21981 raft_consensus.cc:348] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.007566 21981 raft_consensus.cc:374] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:06.007753 21981 raft_consensus.cc:729] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956d1d86903d4be087a9b0507b1f3954, State: Initialized, Role: FOLLOWER
I20260926 20:50:06.008020 21833 heartbeater.cc:503] Master 127.5.78.62:33207 was elected leader, sending a full tablet report...
I20260926 20:50:06.008736 21982 ts_tablet_manager.cc:1428] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260926 20:50:06.008641 21981 consensus_queue.cc:260] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.010115 21964 heartbeater.cc:503] Master 127.5.78.62:33207 was elected leader, sending a full tablet report...
I20260926 20:50:06.011018 21981 ts_tablet_manager.cc:1428] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260926 20:50:06.013298 21980 ts_tablet_manager.cc:1428] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
W20260926 20:50:06.118696 21965 tablet.cc:2369] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:06.134687 21987 raft_consensus.cc:482] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:50:06.135046 21987 raft_consensus.cc:504] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.136888 21987 leader_election.cc:288] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2:33895), 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535)
I20260926 20:50:06.143908 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.137424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:36965 (local address 127.5.78.3:33535)
0926 20:50:06.137664 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:06.137673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:06.138532 (+ 859us) server_negotiation.cc:407] Connection header received
0926 20:50:06.138704 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:06.138709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:06.138770 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:06.138871 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:06.139541 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.140248 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.142012 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.142915 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.143069 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.143193 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:06.143483 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 20:50:06.143614 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:50:06.143956 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.137207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:48823 (local address 127.5.78.2:33895)
0926 20:50:06.137446 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:06.137454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:06.138396 (+ 942us) server_negotiation.cc:407] Connection header received
0926 20:50:06.138656 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:06.138668 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:06.138729 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:06.138830 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:06.139417 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.140139 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.142002 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.142859 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.143025 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.143145 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:06.143512 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 20:50:06.143669 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:50:06.144021 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.137344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:33535 (local address 127.5.78.1:36965)
0926 20:50:06.138375 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.138389 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:06.138548 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:06.138882 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:06.138887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:06.138908 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:06.139388 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:06.139397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.140386 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.140391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:06.141853 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:06.141865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.143159 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.143166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:06.143322 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.143345 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:50:06.143445 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 20:50:06.143587 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"mutex_wait_us":48,"thread_start_us":723,"threads_started":1}
I20260926 20:50:06.144773 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.137165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:33895 (local address 127.5.78.1:48823)
0926 20:50:06.138278 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.138294 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:06.138514 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:06.138845 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:06.138849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:06.138870 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:06.139265 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:06.139274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.140284 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.140290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:06.141837 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:06.141849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.143110 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.143116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:06.143280 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.143305 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:50:06.144217 (+ 912us) client_negotiation.cc:240] Negotiation successful
0926 20:50:06.144353 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":4,"thread_start_us":736,"threads_started":1}
W20260926 20:50:06.144970 21834 tablet.cc:2369] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:06.145694 21919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ea734c5b574c4db67ddc6b5cd6618f" candidate_uuid: "11ed9f30ff5b4db4b231148a0307f672" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d1d86903d4be087a9b0507b1f3954" is_pre_election: true
I20260926 20:50:06.145983 21788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ea734c5b574c4db67ddc6b5cd6618f" candidate_uuid: "11ed9f30ff5b4db4b231148a0307f672" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "553ddf9a6bec41d7918f563d808128c3" is_pre_election: true
I20260926 20:50:06.146368 21919 raft_consensus.cc:2454] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11ed9f30ff5b4db4b231148a0307f672 in term 0.
I20260926 20:50:06.146523 21788 raft_consensus.cc:2454] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11ed9f30ff5b4db4b231148a0307f672 in term 0.
I20260926 20:50:06.147194 21590 leader_election.cc:302] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11ed9f30ff5b4db4b231148a0307f672, 553ddf9a6bec41d7918f563d808128c3; no voters:
I20260926 20:50:06.147635 21987 raft_consensus.cc:2781] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:50:06.147789 21987 raft_consensus.cc:482] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:50:06.147895 21987 raft_consensus.cc:3037] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:06.150149 21987 raft_consensus.cc:504] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.150934 21987 leader_election.cc:288] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [CANDIDATE]: Term 1 election: Requested vote from peers 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2:33895), 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535)
I20260926 20:50:06.151533 21788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ea734c5b574c4db67ddc6b5cd6618f" candidate_uuid: "11ed9f30ff5b4db4b231148a0307f672" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "553ddf9a6bec41d7918f563d808128c3"
I20260926 20:50:06.151571 21919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ea734c5b574c4db67ddc6b5cd6618f" candidate_uuid: "11ed9f30ff5b4db4b231148a0307f672" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956d1d86903d4be087a9b0507b1f3954"
I20260926 20:50:06.151750 21788 raft_consensus.cc:3037] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:06.151865 21919 raft_consensus.cc:3037] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 0 FOLLOWER]: Advancing to term 1
W20260926 20:50:06.152988 21702 tablet.cc:2369] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:06.154246 21919 raft_consensus.cc:2454] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11ed9f30ff5b4db4b231148a0307f672 in term 1.
I20260926 20:50:06.154624 21788 raft_consensus.cc:2454] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11ed9f30ff5b4db4b231148a0307f672 in term 1.
I20260926 20:50:06.154790 21592 leader_election.cc:302] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11ed9f30ff5b4db4b231148a0307f672, 956d1d86903d4be087a9b0507b1f3954; no voters:
I20260926 20:50:06.155172 21987 raft_consensus.cc:2781] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:06.155655 21987 raft_consensus.cc:686] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 1 LEADER]: Becoming Leader. State: Replica: 11ed9f30ff5b4db4b231148a0307f672, State: Running, Role: LEADER
I20260926 20:50:06.156199 21987 consensus_queue.cc:237] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } }
I20260926 20:50:06.160888 21515 catalog_manager.cc:5495] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1). New cstate: current_term: 1 leader_uuid: "11ed9f30ff5b4db4b231148a0307f672" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11ed9f30ff5b4db4b231148a0307f672" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 42711 } health_report { overall_health: HEALTHY } } }
I20260926 20:50:06.203908 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.198286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.5.78.1:42711)
0926 20:50:06.198524 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:06.198530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:06.198550 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:50:06.198702 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:06.198707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:06.198786 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:06.198955 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:50:06.199348 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.200095 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.201572 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.201995 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.202139 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.202743 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:50:06.203247 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:50:06.203423 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:06.203473 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 20:50:06.203583 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:50:06.212288 21788 raft_consensus.cc:1261] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Refusing update from remote peer 11ed9f30ff5b4db4b231148a0307f672: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:06.212347 21919 raft_consensus.cc:1261] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Refusing update from remote peer 11ed9f30ff5b4db4b231148a0307f672: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:06.213325 21991 consensus_queue.cc:1035] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "553ddf9a6bec41d7918f563d808128c3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:06.213673 21987 consensus_queue.cc:1035] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "956d1d86903d4be087a9b0507b1f3954" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:06.226904 21995 mvcc.cc:205] Tried to move back new op lower bound from 7333706982235439104 to 7333706982019907584. Current Snapshot: MvccSnapshot[applied={T|T < 7333706982235439104}]
I20260926 20:50:06.240185 21515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:06.240468 21515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:06.244138 21515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Sending DeleteTablet for 3 replicas of tablet d6ea734c5b574c4db67ddc6b5cd6618f
I20260926 20:50:06.245429 21899 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ea734c5b574c4db67ddc6b5cd6618f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:06 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:36402
I20260926 20:50:06.245537 21768 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ea734c5b574c4db67ddc6b5cd6618f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:06 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:34018
I20260926 20:50:06.245589 21636 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ea734c5b574c4db67ddc6b5cd6618f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:06 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:46986
I20260926 20:50:06.246490 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21553
I20260926 20:50:06.259055 22004 tablet_replica.cc:330] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: stopping tablet replica
I20260926 20:50:06.259835 22004 raft_consensus.cc:2229] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:06.260560 22004 raft_consensus.cc:2258] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:06.261356 22006 tablet_replica.cc:330] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: stopping tablet replica
I20260926 20:50:06.264976 22004 ts_tablet_manager.cc:1905] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:06.262362 22006 raft_consensus.cc:2229] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:50:06.266052 22006 raft_consensus.cc:2258] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:06.269078 22006 ts_tablet_manager.cc:1905] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:06.270294 22005 tablet_replica.cc:330] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: stopping tablet replica
I20260926 20:50:06.270807 22005 raft_consensus.cc:2229] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:06.271347 22005 raft_consensus.cc:2258] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:06.269639 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.248501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.5.78.1:42711)
0926 20:50:06.248685 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:06.248691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:06.248710 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:50:06.248771 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:06.248776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:06.248849 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:06.248956 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:06.251025 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.252321 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.254358 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.255068 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.255217 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.260235 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:06.260253 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:06.260274 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:06.260338 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:06.264836 (+ 4498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:06.266041 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:06.266051 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:06.266054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:06.266316 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:06.266838 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:06.266843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:06.266847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:06.267343 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:06.267663 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:06.268935 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0926 20:50:06.269205 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:50:06.272696 21696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:06.276907 22005 ts_tablet_manager.cc:1905] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:06.277750 22004 ts_tablet_manager.cc:1918] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:06.277940 22004 log.cc:1196] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/d6ea734c5b574c4db67ddc6b5cd6618f
I20260926 20:50:06.278532 22004 ts_tablet_manager.cc:1939] T d6ea734c5b574c4db67ddc6b5cd6618f P 956d1d86903d4be087a9b0507b1f3954: Deleting consensus metadata
I20260926 20:50:06.280776 21503 catalog_manager.cc:4841] TS 956d1d86903d4be087a9b0507b1f3954 (127.5.78.3:33535): tablet d6ea734c5b574c4db67ddc6b5cd6618f (table test-table [id=1c3df298c2754a80ac87002897714780]) successfully deleted
I20260926 20:50:06.283491 22005 ts_tablet_manager.cc:1918] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:06.283704 22005 log.cc:1196] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/d6ea734c5b574c4db67ddc6b5cd6618f
I20260926 20:50:06.284168 22005 ts_tablet_manager.cc:1939] T d6ea734c5b574c4db67ddc6b5cd6618f P 553ddf9a6bec41d7918f563d808128c3: Deleting consensus metadata
I20260926 20:50:06.285763 21501 catalog_manager.cc:4841] TS 553ddf9a6bec41d7918f563d808128c3 (127.5.78.2:33895): tablet d6ea734c5b574c4db67ddc6b5cd6618f (table test-table [id=1c3df298c2754a80ac87002897714780]) successfully deleted
I20260926 20:50:06.375545 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21553
W20260926 20:50:06.392722 21501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.78.1:42711 (error 108)
I20260926 20:50:06.393573 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21705
I20260926 20:50:06.395249 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.394739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47014)
0926 20:50:06.394947 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.395074 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 20:50:06.395479 21501 catalog_manager.cc:4584] TS 11ed9f30ff5b4db4b231148a0307f672 (127.5.78.1:42711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6ea734c5b574c4db67ddc6b5cd6618f: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
I20260926 20:50:06.412768 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.394410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34026 (local address 127.5.78.2:33895)
0926 20:50:06.394645 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:06.394651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:06.394673 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:50:06.394907 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:06.394915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:06.395015 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:06.395136 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:06.396648 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.397414 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.398926 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.399328 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.399477 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.406366 (+ 6889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:06.406380 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:06.406394 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:06.406495 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:06.409670 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:06.410556 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:06.410563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:06.410566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:06.410707 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:06.411096 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:06.411106 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:06.411108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:06.411488 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:06.411789 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:06.412161 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:50:06.412392 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:50:06.413442 21828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:06.449401 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.448859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47028)
0926 20:50:06.449105 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.449223 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260926 20:50:06.450619 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.450150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47038)
0926 20:50:06.450342 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.450474 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 20:50:06.503914 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21705
I20260926 20:50:06.511178 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.510712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47052)
0926 20:50:06.510947 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.511030 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 20:50:06.512373 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.512018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47058)
0926 20:50:06.512171 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.512251 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 20:50:06.521610 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21836
I20260926 20:50:06.535888 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.522451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.5.78.3:33535)
0926 20:50:06.522638 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:06.522647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:06.522669 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:50:06.522903 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:06.522909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:06.522985 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:06.523097 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:06.524380 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.525157 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.526572 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:06.526953 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:06.527097 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:06.529709 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:06.529727 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:06.529742 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:06.529780 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:06.532661 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:06.533502 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:06.533512 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:06.533514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:06.533662 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:06.534079 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:06.534084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:06.534090 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:06.534639 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:06.534950 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:06.535286 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 20:50:06.535521 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 20:50:06.536608 21959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:06.593079 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.592476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47074)
0926 20:50:06.592745 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.592857 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260926 20:50:06.594345 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:06.593876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:42711 (local address 127.0.0.1:47082)
0926 20:50:06.594104 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:06.594201 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 20:50:06.626798 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21836
I20260926 20:50:06.643254 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 21483
I20260926 20:50:06.644510 21546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:06.759361 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 21483
I20260926 20:50:26.782089 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33207
--webserver_interface=127.5.78.62
--webserver_port=41625
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:27.130635 22012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:27.130941 22012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:27.131120 22012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:27.138470 22012 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 20:50:27.138576 22012 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 20:50:27.138652 22012 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 20:50:27.138727 22012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:50:27.138824 22012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:27.138953 22012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:27.139014 22012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:50:27.139062 22012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:50:27.149149 22012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=41625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:27.150539 22012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:27.151913 22012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:27.164765 22018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:27.164877 22020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:27.164882 22017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:27.165823 22012 server_base.cc:1034] running on GCE node
I20260926 20:50:27.166657 22012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:27.168056 22012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:27.169283 22012 hybrid_clock.cc:648] HybridClock initialized: now 1790455827169222 us; error 31 us; skew 500 ppm
I20260926 20:50:27.169588 22012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:27 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455827, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:27.176105 22012 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:50:27.177723 22012 webserver.cc:460] Webserver started at http://127.5.78.62:41625/ using document root <none> and password file <none>
I20260926 20:50:27.178561 22012 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:27.178686 22012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:27.184561 22012 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:50:27.188061 22027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:27.189685 22012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:50:27.189916 22012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ed6c1a3a012e455a9019cb0648ffee8e"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "8f72886d5eb4f94cd587bdb17066fbbd"
server_key_iv: "5d8abeb948361140a80bf2900a33fbd0"
server_key_version: "encryptionkey@0"
I20260926 20:50:27.191061 22012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:27.219277 22012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:27.224181 22012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:27.224505 22012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:27.243019 22012 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33207
I20260926 20:50:27.243120 22078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33207 every 8 connection(s)
I20260926 20:50:27.244663 22012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:50:27.247164 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22012
I20260926 20:50:27.248025 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:42711
--local_ip_for_outbound_sockets=127.5.78.1
--tserver_master_addrs=127.5.78.62:33207
--webserver_port=35893
--webserver_interface=127.5.78.1
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:50:27.252532 22079 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:50:27.256664 22079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap starting.
I20260926 20:50:27.290944 22079 log.cc:824] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:27.313725 22079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:50:27.314550 22079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Bootstrap complete.
I20260926 20:50:27.331259 22079 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:27.332317 22079 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed6c1a3a012e455a9019cb0648ffee8e, State: Initialized, Role: FOLLOWER
I20260926 20:50:27.333070 22079 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:27.333338 22079 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:50:27.333475 22079 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:50:27.333673 22079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 2 FOLLOWER]: Advancing to term 3
I20260926 20:50:27.337714 22079 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:27.338271 22079 leader_election.cc:302] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed6c1a3a012e455a9019cb0648ffee8e; no voters:
I20260926 20:50:27.339012 22079 leader_election.cc:288] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 20:50:27.339233 22083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 3 FOLLOWER]: Leader election won for term 3
I20260926 20:50:27.340262 22083 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [term 3 LEADER]: Becoming Leader. State: Replica: ed6c1a3a012e455a9019cb0648ffee8e, State: Running, Role: LEADER
I20260926 20:50:27.341132 22079 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:50:27.340978 22083 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } }
I20260926 20:50:27.344406 22085 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed6c1a3a012e455a9019cb0648ffee8e. Latest consensus state: current_term: 3 leader_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } } }
I20260926 20:50:27.344839 22085 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:27.345474 22084 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6c1a3a012e455a9019cb0648ffee8e" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33207 } } }
I20260926 20:50:27.345804 22084 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:27.348256 22090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:50:27.359707 22090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:50:27.361970 22090 catalog_manager.cc:1259] Loaded cluster ID: 084db432593742b2ac4231a485d2de38
I20260926 20:50:27.362210 22090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:50:27.368011 22090 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:50:27.370595 22090 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Loaded TSK: 0
I20260926 20:50:27.370890 22090 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed6c1a3a012e455a9019cb0648ffee8e: Loaded TSK: 1
F20260926 20:50:27.372633 22090 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f734e73e9ad google::LogMessage::Fail() at ??:0
@ 0x7f734e74188a google::LogMessage::SendToLog() at ??:0
@ 0x7f734e73e4de google::LogMessage::Flush() at ??:0
@ 0x7f734e73fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260926 20:50:27.384191 22102 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f73524a27d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f735249173e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f73641711c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f7364170783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f73641fc2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f73641fc0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f73642465e9 std::function<>::operator()() at ??:0
@ 0x7f7364174cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f7364172c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f73641fa610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f73641fa439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f735138a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f735138d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f735138d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7351368eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f73546426db start_thread at ??:0
@ 0x7f734d5d861f clone at ??:0
W20260926 20:50:27.670260 22081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:27.670650 22081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:27.670835 22081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:27.678304 22081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:27.678491 22081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:27.678679 22081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:50:27.688448 22081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:42711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=35893
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:27.689810 22081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:27.691183 22081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:27.704869 22130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:27.704797 22128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:27.704849 22127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:27.705586 22081 server_base.cc:1034] running on GCE node
I20260926 20:50:27.706218 22081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:27.707309 22081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:27.708596 22081 hybrid_clock.cc:648] HybridClock initialized: now 1790455827708524 us; error 51 us; skew 500 ppm
I20260926 20:50:27.709002 22081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:27 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455827, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:27.712637 22081 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:50:27.714200 22081 webserver.cc:460] Webserver started at http://127.5.78.1:35893/ using document root <none> and password file <none>
I20260926 20:50:27.715031 22081 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:27.715194 22081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:27.720912 22081 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 20:50:27.724339 22137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:27.725901 22081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:50:27.726112 22081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "11ed9f30ff5b4db4b231148a0307f672"
format_stamp: "Formatted at 2026-09-26 20:49:41 on dist-test-slave-z7j8"
server_key: "d6bd48b92f8fd929fd9e3d05c7d7a731"
server_key_iv: "0314068992a1f1ecb9f79c65d3024a20"
server_key_version: "encryptionkey@0"
I20260926 20:50:27.727237 22081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:27.744551 22081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:27.749078 22081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:27.749385 22081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:27.750552 22081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:27.752485 22144 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260926 20:50:27.757866 22144 ts_tablet_manager.cc:1852] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 20:50:27.758988 22144 ts_tablet_manager.cc:1905] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:27.766633 22144 ts_tablet_manager.cc:1918] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:27.766870 22144 log.cc:1196] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/d6ea734c5b574c4db67ddc6b5cd6618f
I20260926 20:50:27.767421 22144 ts_tablet_manager.cc:1939] T d6ea734c5b574c4db67ddc6b5cd6618f P 11ed9f30ff5b4db4b231148a0307f672: Deleting consensus metadata
I20260926 20:50:27.767851 22081 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 20:50:27.768033 22081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260926 20:50:27.768193 22081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:27.768290 22081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:27.802935 22081 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:42711
I20260926 20:50:27.802997 22250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:42711 every 8 connection(s)
I20260926 20:50:27.804346 22081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:50:27.809481 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:27.807599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:56181)
0926 20:50:27.808363 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:27.808573 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":298,"threads_started":1}
W20260926 20:50:27.810256 22251 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:33207 (0 consecutive failures): Network error: Failed to ping master at 127.5.78.62:33207: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
I20260926 20:50:27.810801 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22081
I20260926 20:50:27.811307 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:27.810934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:50411)
0926 20:50:27.811101 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:27.811189 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 20:50:27.811561 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:33895
--local_ip_for_outbound_sockets=127.5.78.2
--tserver_master_addrs=127.5.78.62:33207
--webserver_port=37003
--webserver_interface=127.5.78.2
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:50:27.812281 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:27.811922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:47881)
0926 20:50:27.812066 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:27.812158 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 20:50:27.812542 22251 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:50:28.153581 22254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:28.153888 22254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:28.154106 22254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:28.161589 22254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:28.161724 22254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:28.161950 22254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:50:28.172653 22254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:33895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=37003
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:28.174012 22254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:28.175431 22254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:28.189368 22262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:28.189428 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:28.189445 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:28.190016 22254 server_base.cc:1034] running on GCE node
I20260926 20:50:28.190801 22254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:28.191735 22254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:28.192931 22254 hybrid_clock.cc:648] HybridClock initialized: now 1790455828192876 us; error 44 us; skew 500 ppm
I20260926 20:50:28.193266 22254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:28 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:28.196485 22254 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:50:28.197872 22254 webserver.cc:460] Webserver started at http://127.5.78.2:37003/ using document root <none> and password file <none>
I20260926 20:50:28.198575 22254 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:28.198679 22254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:28.203907 22254 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 20:50:28.206797 22269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:28.208178 22254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:50:28.208359 22254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "553ddf9a6bec41d7918f563d808128c3"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "49443357b90c71a01bf1e56628ba84e3"
server_key_iv: "d66a4620187812fc61ed14989a2c8fd9"
server_key_version: "encryptionkey@0"
I20260926 20:50:28.209353 22254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:28.233683 22254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:28.238529 22254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:28.238855 22254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:28.239888 22254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:28.241240 22254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:28.241341 22254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:28.241415 22254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:28.241469 22254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:28.275252 22254 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:33895
I20260926 20:50:28.275347 22381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:33895 every 8 connection(s)
I20260926 20:50:28.276691 22254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:50:28.281720 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.279882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:42133)
0926 20:50:28.280610 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.280834 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":294,"threads_started":1}
W20260926 20:50:28.282459 22382 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:33207 (0 consecutive failures): Network error: Failed to ping master at 127.5.78.62:33207: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
I20260926 20:50:28.283288 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.282942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:34961)
0926 20:50:28.283095 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.283170 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 20:50:28.284179 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.283812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:41229)
0926 20:50:28.283982 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.284072 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 20:50:28.284415 22382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:50:28.286108 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22254
I20260926 20:50:28.286818 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:33535
--local_ip_for_outbound_sockets=127.5.78.3
--tserver_master_addrs=127.5.78.62:33207
--webserver_port=42563
--webserver_interface=127.5.78.3
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:28.614805 22385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:28.615114 22385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:28.615268 22385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:28.622675 22385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:28.622807 22385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:28.622999 22385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:50:28.633200 22385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:33535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=42563
--tserver_master_addrs=127.5.78.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:28.634662 22385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:28.636066 22385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:28.650331 22391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:28.650507 22390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:28.650359 22393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:28.651003 22385 server_base.cc:1034] running on GCE node
I20260926 20:50:28.651707 22385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:28.652806 22385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:28.654074 22385 hybrid_clock.cc:648] HybridClock initialized: now 1790455828654025 us; error 47 us; skew 500 ppm
I20260926 20:50:28.654520 22385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:28 dist-test-slave-z7j8 krb5kdc[20931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:28.657847 22385 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:50:28.659477 22385 webserver.cc:460] Webserver started at http://127.5.78.3:42563/ using document root <none> and password file <none>
I20260926 20:50:28.660200 22385 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:28.660353 22385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:28.665913 22385 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 20:50:28.668821 22400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:28.670331 22385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 20:50:28.670553 22385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "956d1d86903d4be087a9b0507b1f3954"
format_stamp: "Formatted at 2026-09-26 20:49:42 on dist-test-slave-z7j8"
server_key: "64164f54c6397e50b98aed0691dbc537"
server_key_iv: "5756214e0068d53078fed9e567a28935"
server_key_version: "encryptionkey@0"
I20260926 20:50:28.671561 22385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:28.696587 22385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:28.700877 22385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:28.701174 22385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:28.702179 22385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:28.703656 22385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:28.703784 22385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:28.703918 22385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:28.704006 22385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:28.739604 22385 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:33535
I20260926 20:50:28.739703 22512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:33535 every 8 connection(s)
I20260926 20:50:28.741101 22385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:50:28.746183 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.744294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:41329)
0926 20:50:28.745056 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.745280 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":296,"threads_started":1}
W20260926 20:50:28.746889 22513 heartbeater.cc:644] Failed to heartbeat to 127.5.78.62:33207 (0 consecutive failures): Network error: Failed to ping master at 127.5.78.62:33207: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
I20260926 20:50:28.747750 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.747366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:60521)
0926 20:50:28.747558 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.747635 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 20:50:28.748649 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.748271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:39939)
0926 20:50:28.748424 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.748529 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 20:50:28.748894 22513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:50:28.751045 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22385
I20260926 20:50:28.814195 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:28.813251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:46745)
0926 20:50:28.813818 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:28.813920 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":286,"threads_started":1}
I20260926 20:50:29.285773 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:29.284971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:50455)
0926 20:50:29.285437 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:29.285537 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260926 20:50:29.750334 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:29.749470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:42839)
0926 20:50:29.749985 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:29.750087 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":271,"threads_started":1}
I20260926 20:50:29.816066 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:29.815345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:47407)
0926 20:50:29.815797 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:29.815885 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":244,"threads_started":1}
I20260926 20:50:30.287915 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:30.286922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:45191)
0926 20:50:30.287560 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:30.287675 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":297,"threads_started":1}
I20260926 20:50:30.752180 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:30.751392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:42065)
0926 20:50:30.751909 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:30.751998 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":270,"threads_started":1}
I20260926 20:50:30.817860 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:30.817106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:40473)
0926 20:50:30.817588 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:30.817683 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":240,"threads_started":1}
I20260926 20:50:31.289660 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:31.288889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:58797)
0926 20:50:31.289354 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:31.289448 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
I20260926 20:50:31.754130 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:31.753334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:54457)
0926 20:50:31.753841 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:31.753944 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":269,"threads_started":1}
I20260926 20:50:31.819551 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:31.818817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:53687)
0926 20:50:31.819306 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:31.819388 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":268,"threads_started":1}
I20260926 20:50:32.291532 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:32.290739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:37825)
0926 20:50:32.291228 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:32.291324 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":219,"threads_started":1}
I20260926 20:50:32.755828 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:32.755030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:38157)
0926 20:50:32.755546 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:32.755639 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":264,"threads_started":1}
I20260926 20:50:32.821470 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:32.820599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:51923)
0926 20:50:32.821161 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:32.821253 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":272,"threads_started":1}
I20260926 20:50:33.293926 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:33.292705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:53033)
0926 20:50:33.293587 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:33.293694 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":586,"threads_started":1}
I20260926 20:50:33.757514 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:33.756740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:42635)
0926 20:50:33.757254 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:33.757341 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":259,"threads_started":1}
I20260926 20:50:33.823120 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:33.822375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:36721)
0926 20:50:33.822870 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:33.822958 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":233,"threads_started":1}
I20260926 20:50:34.295610 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:34.294879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:50287)
0926 20:50:34.295322 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:34.295434 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
I20260926 20:50:34.759459 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:34.758562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:47693)
0926 20:50:34.759133 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:34.759274 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":323,"threads_started":1}
I20260926 20:50:34.825134 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:34.824236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:39487)
0926 20:50:34.824815 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:34.824919 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":278,"threads_started":1}
I20260926 20:50:35.297864 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:35.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:51773)
0926 20:50:35.297460 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:35.297564 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":284,"threads_started":1}
I20260926 20:50:35.761360 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:35.760530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:49777)
0926 20:50:35.761063 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:35.761150 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":219,"threads_started":1}
I20260926 20:50:35.826894 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:35.826113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:33941)
0926 20:50:35.826639 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:35.826726 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":316,"threads_started":1}
I20260926 20:50:36.299484 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:36.298812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:51363)
0926 20:50:36.299243 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:36.299323 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":221,"threads_started":1}
I20260926 20:50:36.763098 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:36.762338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:54675)
0926 20:50:36.762853 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:36.762938 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":305,"threads_started":1}
I20260926 20:50:36.828691 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:36.827885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:44491)
0926 20:50:36.828401 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:36.828524 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":263,"threads_started":1}
I20260926 20:50:37.301358 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:37.300503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:59475)
0926 20:50:37.301069 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:37.301166 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":275,"threads_started":1}
I20260926 20:50:37.764925 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:37.764096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:59287)
0926 20:50:37.764635 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:37.764723 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":262,"threads_started":1}
I20260926 20:50:37.830380 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:37.829652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:39443)
0926 20:50:37.830133 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:37.830214 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":266,"threads_started":1}
I20260926 20:50:38.303193 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:38.302382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:47849)
0926 20:50:38.302921 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:38.303009 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":261,"threads_started":1}
I20260926 20:50:38.766819 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:38.765987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:48337)
0926 20:50:38.766572 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:38.766661 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":311,"threads_started":1}
I20260926 20:50:38.832218 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:38.831419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:43077)
0926 20:50:38.831932 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:38.832028 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":287,"threads_started":1}
I20260926 20:50:39.305119 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:39.304235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:42929)
0926 20:50:39.304786 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:39.304887 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":275,"threads_started":1}
I20260926 20:50:39.768491 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:39.767759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:60645)
0926 20:50:39.768228 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:39.768314 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":232,"threads_started":1}
I20260926 20:50:39.833951 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:39.833183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:54685)
0926 20:50:39.833695 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:39.833787 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":271,"threads_started":1}
I20260926 20:50:40.307014 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:40.306202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:49333)
0926 20:50:40.306755 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:40.306854 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":299,"threads_started":1}
I20260926 20:50:40.770522 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:40.769611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:39975)
0926 20:50:40.770124 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:40.770236 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":255,"threads_started":1}
I20260926 20:50:40.835749 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:40.834977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:42121)
0926 20:50:40.835455 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:40.835539 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":240,"threads_started":1}
I20260926 20:50:41.308871 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:41.307963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:33775)
0926 20:50:41.308560 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:41.308660 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":314,"threads_started":1}
I20260926 20:50:41.772356 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:41.771552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:59787)
0926 20:50:41.772060 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:41.772163 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":245,"threads_started":1}
I20260926 20:50:41.837644 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:41.836729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:56227)
0926 20:50:41.837319 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:41.837430 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":298,"threads_started":1}
I20260926 20:50:42.311304 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:42.309917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:41659)
0926 20:50:42.310875 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:42.310999 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":293,"threads_started":1}
I20260926 20:50:42.774096 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:42.773372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:55633)
0926 20:50:42.773828 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:42.773909 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260926 20:50:42.839644 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:42.838904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:40637)
0926 20:50:42.839373 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:42.839451 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
I20260926 20:50:43.313369 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:43.312444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.2:42079)
0926 20:50:43.313036 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:43.313123 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":341,"threads_started":1}
I20260926 20:50:43.755293 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22081
I20260926 20:50:43.774895 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:43.756558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.5.78.1:42711)
0926 20:50:43.757175 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:43.757191 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:43.757232 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:50:43.759062 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:43.759074 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:43.759541 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:43.759860 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:43.761404 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:43.762833 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:43.764237 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:43.764954 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:43.765121 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:43.768005 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:43.768014 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:43.768055 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:43.768099 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:43.771283 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:43.772060 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:43.772068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:43.772075 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:43.772269 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:43.772783 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:43.772788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:43.772790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:43.773415 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:43.773678 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:43.774100 (+ 422us) server_negotiation.cc:299] Negotiation successful
0926 20:50:43.774550 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":277,"threads_started":1}
I20260926 20:50:43.775974 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:43.775214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.3:45355)
0926 20:50:43.775724 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:43.775809 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":280,"threads_started":1}
I20260926 20:50:43.776662 22246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:43.864588 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:43.863516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33207 (local address 127.5.78.1:38273)
0926 20:50:43.864075 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:43.864253 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":276,"threads_started":1}
I20260926 20:50:43.866595 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22081
I20260926 20:50:43.885378 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22254
I20260926 20:50:43.903139 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:43.886391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.5.78.2:33895)
0926 20:50:43.886941 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:43.886956 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:43.886994 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:50:43.888602 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:43.888612 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:43.889052 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:43.889363 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:43.890924 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:43.892212 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:43.893368 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:43.894037 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:43.894168 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:43.896730 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:43.896738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:43.896753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:43.896795 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:43.899847 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:43.900601 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:43.900609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:43.900616 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:43.900812 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:43.901178 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:43.901182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:43.901184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:43.901763 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:43.902024 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:43.902464 (+ 440us) server_negotiation.cc:299] Negotiation successful
0926 20:50:43.902841 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":235,"threads_started":1}
I20260926 20:50:43.904524 22377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:43.998063 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22254
I20260926 20:50:44.016569 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22385
I20260926 20:50:44.034256 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:44.017624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.5.78.3:33535)
0926 20:50:44.018225 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:44.018240 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:44.018284 (+ 44us) server_negotiation.cc:407] Connection header received
0926 20:50:44.019986 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:44.020003 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:44.020446 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:44.020750 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:44.022039 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:44.023331 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:44.024636 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:44.025383 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:44.025567 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:44.028115 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:44.028122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:44.028138 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:44.028174 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:44.031193 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:44.031868 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:44.031876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:44.031883 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:44.032078 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:44.032432 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:44.032436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:44.032438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:44.033006 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:44.033237 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:44.033620 (+ 383us) server_negotiation.cc:299] Negotiation successful
0926 20:50:44.033948 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":278,"threads_started":1}
I20260926 20:50:44.035681 22508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:44.123348 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22385
2026-09-26T20:50:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64613 ms)
[----------] 30 tests from SecurityITest (302704 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:50:44 dist-test-slave-z7j8 krb5kdc[22571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:50:44 dist-test-slave-z7j8 krb5kdc[22571](info): set up 2 sockets
Sep 26 20:50:44 dist-test-slave-z7j8 krb5kdc[22571](info): commencing operation
krb5kdc: starting...
W20260926 20:50:45.218847 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:50:45 dist-test-slave-z7j8 krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:50:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:50:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:50:45.474393 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46071
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:46071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:45.811626 22584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:45.811932 22584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:45.812026 22584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:45.819499 22584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:50:45.819623 22584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:45.819670 22584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:45.819720 22584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:50:45.819763 22584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:50:45.829833 22584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:46071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:45.831301 22584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:45.832617 22584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:45.845410 22592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:45.845697 22584 server_base.cc:1034] running on GCE node
W20260926 20:50:45.845418 22590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:45.845992 22589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:45.847078 22584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:45.848779 22584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:45.850066 22584 hybrid_clock.cc:648] HybridClock initialized: now 1790455845849983 us; error 69 us; skew 500 ppm
I20260926 20:50:45.850561 22584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:45 dist-test-slave-z7j8 krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455845, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:45.859709 22584 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:50:45.861521 22584 webserver.cc:460] Webserver started at http://127.5.78.62:43083/ using document root <none> and password file <none>
I20260926 20:50:45.862360 22584 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:45.862583 22584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:45.862938 22584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:45.866103 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "c9494382ccb44f4e93485faf6e73a6c7"
format_stamp: "Formatted at 2026-09-26 20:50:45 on dist-test-slave-z7j8"
server_key: "57e529be6c09c683d2c12dca4d5e3092"
server_key_iv: "216e0230b2f2b0b215f2c44d4d13d7a5"
server_key_version: "encryptionkey@0"
I20260926 20:50:45.866904 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "c9494382ccb44f4e93485faf6e73a6c7"
format_stamp: "Formatted at 2026-09-26 20:50:45 on dist-test-slave-z7j8"
server_key: "57e529be6c09c683d2c12dca4d5e3092"
server_key_iv: "216e0230b2f2b0b215f2c44d4d13d7a5"
server_key_version: "encryptionkey@0"
I20260926 20:50:45.872282 22584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:50:45.875905 22599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:45.877234 22584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:50:45.877422 22584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c9494382ccb44f4e93485faf6e73a6c7"
format_stamp: "Formatted at 2026-09-26 20:50:45 on dist-test-slave-z7j8"
server_key: "57e529be6c09c683d2c12dca4d5e3092"
server_key_iv: "216e0230b2f2b0b215f2c44d4d13d7a5"
server_key_version: "encryptionkey@0"
I20260926 20:50:45.877656 22584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:45.891125 22584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:45.895761 22584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:45.896065 22584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:45.915762 22584 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:46071
I20260926 20:50:45.915853 22650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:46071 every 8 connection(s)
I20260926 20:50:45.917202 22584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:50:45.918700 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22584
I20260926 20:50:45.918913 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:50:45.919745 5432 external_mini_cluster.cc:1428] Setting key 7dcf03944623eca9f8eb07e067741ab8
I20260926 20:50:45.922464 22651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:50:45 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:45.936757 22651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7: Bootstrap starting.
I20260926 20:50:45.941416 22651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:45.942766 22651 log.cc:824] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:45.946667 22651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7: No bootstrap required, opened a new log
I20260926 20:50:45.946874 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:45.921987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.5.78.62:46071)
0926 20:50:45.922877 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:45.922890 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:45.922935 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:50:45.924673 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:45.924697 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:45.925129 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:45.925493 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:45.927596 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:45.929040 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:45.930987 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:45.931723 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:45.931873 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:45.937105 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:45.937119 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:45.937144 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:45.937204 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:45.941429 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:45.942446 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:45.942457 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:45.942468 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:45.942733 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:45.943276 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:45.943281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:45.943286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:45.944029 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:45.944320 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:45.944882 (+ 562us) server_negotiation.cc:299] Negotiation successful
0926 20:50:45.945373 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":429,"threads_started":1}
I20260926 20:50:45.954710 22651 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9494382ccb44f4e93485faf6e73a6c7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46071 } }
I20260926 20:50:45.955132 22651 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:45.955216 22651 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9494382ccb44f4e93485faf6e73a6c7, State: Initialized, Role: FOLLOWER
I20260926 20:50:45.955772 22651 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9494382ccb44f4e93485faf6e73a6c7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46071 } }
I20260926 20:50:45.955976 22651 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:50:45.956163 22651 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:50:45.956357 22651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:45.958752 22651 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9494382ccb44f4e93485faf6e73a6c7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46071 } }
I20260926 20:50:45.959277 22651 leader_election.cc:302] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9494382ccb44f4e93485faf6e73a6c7; no voters:
I20260926 20:50:45.959874 22651 leader_election.cc:288] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:50:45.960039 22656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:45.960777 22656 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [term 1 LEADER]: Becoming Leader. State: Replica: c9494382ccb44f4e93485faf6e73a6c7, State: Running, Role: LEADER
I20260926 20:50:45.961387 22656 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9494382ccb44f4e93485faf6e73a6c7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46071 } }
I20260926 20:50:45.962050 22651 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:50:45.964437 22658 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9494382ccb44f4e93485faf6e73a6c7. Latest consensus state: current_term: 1 leader_uuid: "c9494382ccb44f4e93485faf6e73a6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9494382ccb44f4e93485faf6e73a6c7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46071 } } }
I20260926 20:50:45.964913 22658 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:45.965153 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9494382ccb44f4e93485faf6e73a6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9494382ccb44f4e93485faf6e73a6c7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46071 } } }
I20260926 20:50:45.965444 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7 [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:45.967373 22664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:50:45.972306 22664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:50:45.980558 22664 catalog_manager.cc:1347] Generated new cluster ID: e5ce97769b644d6a8087b5ce514c4c4a
I20260926 20:50:45.980703 22664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:50:45.990477 22664 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:50:45.992381 22664 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:50:46.000990 22664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7: Generated new TSK 0
I20260926 20:50:46.002058 22664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:50:46.062274 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46071
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:46.403923 22677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:46.404266 22677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:46.404450 22677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:46.412011 22677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:46.412217 22677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:46.412420 22677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:50:46.422611 22677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:46.424015 22677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:46.425366 22677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:46.439124 22683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:46.439128 22682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:46.439158 22685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:46.439978 22677 server_base.cc:1034] running on GCE node
I20260926 20:50:46.440768 22677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:46.441807 22677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:46.443100 22677 hybrid_clock.cc:648] HybridClock initialized: now 1790455846443054 us; error 50 us; skew 500 ppm
I20260926 20:50:46.443507 22677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:46 dist-test-slave-z7j8 krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455846, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:46.447041 22677 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:50:46.448851 22677 webserver.cc:460] Webserver started at http://127.5.78.1:33181/ using document root <none> and password file <none>
I20260926 20:50:46.449657 22677 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:46.449821 22677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:46.450192 22677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:46.453447 22677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "1149b63b50e84d529ec9264973d575ad"
format_stamp: "Formatted at 2026-09-26 20:50:46 on dist-test-slave-z7j8"
server_key: "775ac5ea6a853f8e2950393cfa581955"
server_key_iv: "ca2eb9395a24cf1cfe8a83d07cf3376d"
server_key_version: "encryptionkey@0"
I20260926 20:50:46.454278 22677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "1149b63b50e84d529ec9264973d575ad"
format_stamp: "Formatted at 2026-09-26 20:50:46 on dist-test-slave-z7j8"
server_key: "775ac5ea6a853f8e2950393cfa581955"
server_key_iv: "ca2eb9395a24cf1cfe8a83d07cf3376d"
server_key_version: "encryptionkey@0"
I20260926 20:50:46.459789 22677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:50:46.463812 22692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:46.465229 22677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:50:46.465427 22677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "1149b63b50e84d529ec9264973d575ad"
format_stamp: "Formatted at 2026-09-26 20:50:46 on dist-test-slave-z7j8"
server_key: "775ac5ea6a853f8e2950393cfa581955"
server_key_iv: "ca2eb9395a24cf1cfe8a83d07cf3376d"
server_key_version: "encryptionkey@0"
I20260926 20:50:46.465654 22677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:46.490110 22677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:46.495164 22677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:46.495507 22677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:46.496573 22677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:46.498034 22677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:46.498140 22677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:46.498294 22677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:46.498366 22677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:46.534530 22677 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:35465
I20260926 20:50:46.534575 22804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:35465 every 8 connection(s)
I20260926 20:50:46.535921 22677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:50:46.540410 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22677
I20260926 20:50:46.540690 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:50:46.541640 5432 external_mini_cluster.cc:1428] Setting key 5d70efc040af15a4037a1316d072337f
Sep 26 20:50:46 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455846, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:46.563305 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:46.538817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:38335 (local address 127.5.78.62:46071)
0926 20:50:46.539314 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:46.539319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:46.540843 (+ 1524us) server_negotiation.cc:407] Connection header received
0926 20:50:46.543473 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:46.543478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:46.543547 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:46.543647 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:46.545968 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:46.546802 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:46.549590 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:46.550036 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:46.550157 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:46.555349 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:46.555359 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:46.555367 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:46.555420 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:46.559231 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:46.560134 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:46.560143 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:46.560150 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:46.560309 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:46.560827 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:46.560835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:46.560842 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:46.561240 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:46.561408 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:46.562477 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0926 20:50:46.562808 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260926 20:50:46.564112 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:46.539443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46071 (local address 127.5.78.1:38335)
0926 20:50:46.540491 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:46.540553 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:46.543045 (+ 2492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:46.543963 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:46.543977 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:46.544783 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:46.545735 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:46.545771 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:46.546989 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:46.547007 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:46.549356 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:46.549380 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:46.550331 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:46.550343 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:46.550794 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:46.551685 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:46.551727 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:46.555038 (+ 3311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:46.559432 (+ 4394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:46.559440 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:46.559456 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:46.559984 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:46.560453 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:46.560458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:46.560460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:46.560704 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:46.561429 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:46.561436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:46.561924 (+ 488us) client_negotiation.cc:769] Sending connection context
0926 20:50:46.562207 (+ 283us) client_negotiation.cc:240] Negotiation successful
0926 20:50:46.562820 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":392,"threads_started":1}
I20260926 20:50:46.566037 22805 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46071
I20260926 20:50:46.566443 22805 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:46.567327 22805 heartbeater.cc:511] Master 127.5.78.62:46071 requested a full tablet report, sending...
I20260926 20:50:46.570181 22616 ts_manager.cc:194] Registered new tserver with Master: 1149b63b50e84d529ec9264973d575ad (127.5.78.1:35465)
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:50:46.572185 22616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:38335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:50:46.603699 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46071
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:46.942761 22811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:46.943130 22811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:46.943323 22811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:46.950703 22811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:46.950870 22811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:46.951071 22811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:50:46.961174 22811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:46.962857 22811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:46.964323 22811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:46.978992 22819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:46.979112 22816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:46.979091 22817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:46.979930 22811 server_base.cc:1034] running on GCE node
I20260926 20:50:46.980657 22811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:46.981678 22811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:46.982910 22811 hybrid_clock.cc:648] HybridClock initialized: now 1790455846982857 us; error 39 us; skew 500 ppm
I20260926 20:50:46.983263 22811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:46 dist-test-slave-z7j8 krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455846, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:46.986490 22811 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:50:46.988126 22811 webserver.cc:460] Webserver started at http://127.5.78.2:34479/ using document root <none> and password file <none>
I20260926 20:50:46.988786 22811 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:46.988888 22811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:46.989158 22811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:46.992225 22811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "1e1be06cbdc64d59bb26c58bcfac3b52"
format_stamp: "Formatted at 2026-09-26 20:50:46 on dist-test-slave-z7j8"
server_key: "f4af8d39de67d528d7e8fa53664d7c2b"
server_key_iv: "ebf8953ccd159a71081f4800018019b0"
server_key_version: "encryptionkey@0"
I20260926 20:50:46.992988 22811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "1e1be06cbdc64d59bb26c58bcfac3b52"
format_stamp: "Formatted at 2026-09-26 20:50:46 on dist-test-slave-z7j8"
server_key: "f4af8d39de67d528d7e8fa53664d7c2b"
server_key_iv: "ebf8953ccd159a71081f4800018019b0"
server_key_version: "encryptionkey@0"
I20260926 20:50:46.998545 22811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:50:47.002279 22826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:47.003923 22811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:50:47.004135 22811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "1e1be06cbdc64d59bb26c58bcfac3b52"
format_stamp: "Formatted at 2026-09-26 20:50:46 on dist-test-slave-z7j8"
server_key: "f4af8d39de67d528d7e8fa53664d7c2b"
server_key_iv: "ebf8953ccd159a71081f4800018019b0"
server_key_version: "encryptionkey@0"
I20260926 20:50:47.004339 22811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:47.041778 22811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:47.046653 22811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:47.047039 22811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:47.048242 22811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:47.049935 22811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:47.050065 22811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:47.050251 22811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:47.050338 22811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:47.094049 22811 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:33323
I20260926 20:50:47.094110 22938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:33323 every 8 connection(s)
I20260926 20:50:47.095639 22811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:50:47.099924 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22811
I20260926 20:50:47.100099 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:50:47.100666 5432 external_mini_cluster.cc:1428] Setting key de85a713f44dff02fdc2d0794c675601
Sep 26 20:50:47 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455846, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:47.126025 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.098604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:46139 (local address 127.5.78.62:46071)
0926 20:50:47.099098 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:47.099105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:47.100336 (+ 1231us) server_negotiation.cc:407] Connection header received
0926 20:50:47.102634 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:47.102641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:47.102725 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:47.102816 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:47.104352 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.105566 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.107281 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.108097 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.108248 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.115025 (+ 6777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:47.115035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:47.115042 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:47.115100 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:47.121768 (+ 6668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:47.122900 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:47.122908 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:47.122915 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:47.123101 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:47.123658 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:47.123666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:47.123671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:47.124066 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:47.124221 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:47.125253 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0926 20:50:47.125516 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":220,"threads_started":1}
I20260926 20:50:47.128580 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.099228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46071 (local address 127.5.78.2:46139)
0926 20:50:47.100131 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:47.100183 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:47.102315 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:47.103077 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:47.103085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:47.103615 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:47.104172 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.104189 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.105750 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.105754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.107132 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.107145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.108330 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.108335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.108620 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.109505 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:47.109537 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:47.114727 (+ 5190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:47.121960 (+ 7233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:47.121972 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:47.121993 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:47.122721 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:47.123243 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:47.123251 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:47.123276 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:47.123529 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:47.124316 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:47.124326 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:47.124801 (+ 475us) client_negotiation.cc:769] Sending connection context
0926 20:50:47.126273 (+ 1472us) client_negotiation.cc:240] Negotiation successful
0926 20:50:47.126934 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":342,"threads_started":1}
I20260926 20:50:47.131678 22939 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46071
I20260926 20:50:47.132064 22939 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:50:47.133005 22939 heartbeater.cc:511] Master 127.5.78.62:46071 requested a full tablet report, sending...
I20260926 20:50:47.134828 22616 ts_manager.cc:194] Registered new tserver with Master: 1e1be06cbdc64d59bb26c58bcfac3b52 (127.5.78.2:33323)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
I20260926 20:50:47.136302 22616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:46139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:50:47.174479 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46071
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:47.514468 22945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:47.514773 22945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:47.514940 22945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:47.522344 22945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:47.522504 22945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:47.522691 22945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:50:47.533005 22945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:46071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:47.534467 22945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:47.535822 22945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:47.549451 22953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:47.549420 22950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:47.549489 22951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:47.550282 22945 server_base.cc:1034] running on GCE node
I20260926 20:50:47.550952 22945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:47.551893 22945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:47.553118 22945 hybrid_clock.cc:648] HybridClock initialized: now 1790455847553072 us; error 36 us; skew 500 ppm
I20260926 20:50:47.553433 22945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:47 dist-test-slave-z7j8 krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455847, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:47.556751 22945 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:50:47.558338 22945 webserver.cc:460] Webserver started at http://127.5.78.3:35695/ using document root <none> and password file <none>
I20260926 20:50:47.559082 22945 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:47.559211 22945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:47.559532 22945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:47.562547 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "ad048428eaa54a82b59d476e234ce9c6"
format_stamp: "Formatted at 2026-09-26 20:50:47 on dist-test-slave-z7j8"
server_key: "ceb723ae36930fe807f4661e7d6627c7"
server_key_iv: "a62d4a3fb6df23daf775e2902744acba"
server_key_version: "encryptionkey@0"
I20260926 20:50:47.563474 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "ad048428eaa54a82b59d476e234ce9c6"
format_stamp: "Formatted at 2026-09-26 20:50:47 on dist-test-slave-z7j8"
server_key: "ceb723ae36930fe807f4661e7d6627c7"
server_key_iv: "a62d4a3fb6df23daf775e2902744acba"
server_key_version: "encryptionkey@0"
I20260926 20:50:47.569005 22945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:50:47.572729 22960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:47.574208 22945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:50:47.574393 22945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "ad048428eaa54a82b59d476e234ce9c6"
format_stamp: "Formatted at 2026-09-26 20:50:47 on dist-test-slave-z7j8"
server_key: "ceb723ae36930fe807f4661e7d6627c7"
server_key_iv: "a62d4a3fb6df23daf775e2902744acba"
server_key_version: "encryptionkey@0"
I20260926 20:50:47.574671 22945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:47.575878 22805 heartbeater.cc:503] Master 127.5.78.62:46071 was elected leader, sending a full tablet report...
I20260926 20:50:47.586370 22945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:47.590687 22945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:47.590943 22945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:47.592012 22945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:47.593628 22945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:47.593744 22945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:47.593874 22945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:47.593946 22945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:47.629063 22945 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:44325
I20260926 20:50:47.629123 23072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:44325 every 8 connection(s)
I20260926 20:50:47.630692 22945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:50:47.640064 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 22945
I20260926 20:50:47.640215 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:50:47.640689 5432 external_mini_cluster.cc:1428] Setting key e49d09841cb925c22dde4c34574c0ded
Sep 26 20:50:47 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455847, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:47.651851 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.633455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:45049 (local address 127.5.78.62:46071)
0926 20:50:47.633925 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:47.633929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:47.635019 (+ 1090us) server_negotiation.cc:407] Connection header received
0926 20:50:47.636931 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:47.636937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:47.637032 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:47.637158 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:47.638857 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.639718 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.641718 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.642342 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.642515 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.646213 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:47.646218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:47.646221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:47.646254 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:47.649006 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:47.649699 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:47.649703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:47.649705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:47.649821 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:47.650187 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:47.650190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:47.650192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:47.650524 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:47.650641 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:47.651344 (+ 703us) server_negotiation.cc:299] Negotiation successful
0926 20:50:47.651561 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":226,"threads_started":1}
I20260926 20:50:47.652770 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.634058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:46071 (local address 127.5.78.3:45049)
0926 20:50:47.634859 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:47.634898 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:47.636652 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:47.637417 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:47.637426 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:47.637978 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:47.638665 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.638686 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.639915 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.639919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.641579 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.641593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.642627 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.642635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.642966 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.643737 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:47.643774 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:47.646016 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:47.649143 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:47.649150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:47.649164 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:47.649598 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:47.649912 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:47.649915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:47.649917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:47.650098 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:47.650632 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:47.650637 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:47.650975 (+ 338us) client_negotiation.cc:769] Sending connection context
0926 20:50:47.651210 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 20:50:47.651604 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":337,"threads_started":1}
I20260926 20:50:47.654196 23073 heartbeater.cc:347] Connected to a master server at 127.5.78.62:46071
I20260926 20:50:47.654572 23073 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:47.655439 23073 heartbeater.cc:511] Master 127.5.78.62:46071 requested a full tablet report, sending...
I20260926 20:50:47.657035 22616 ts_manager.cc:194] Registered new tserver with Master: ad048428eaa54a82b59d476e234ce9c6 (127.5.78.3:44325)
I20260926 20:50:47.657786 22616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:45049
I20260926 20:50:47.668987 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:50:47.686518 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.673802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.5.78.62:46071)
0926 20:50:47.674035 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:47.674046 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:47.674480 (+ 434us) server_negotiation.cc:407] Connection header received
0926 20:50:47.674640 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:47.674644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:47.674710 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:47.674801 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:47.676255 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.677019 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.678468 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.678936 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.679038 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.681429 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:47.681434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:47.681437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:47.681478 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:47.684113 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:47.684712 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:47.684716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:47.684718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:47.684832 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:47.685162 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:47.685165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:47.685167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:47.685476 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:47.685671 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:47.685948 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 20:50:47.686136 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:50:47.690863 22616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:50:47.697685 22616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:50:47.720361 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.709486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:35465 (local address 127.0.0.1:51154)
0926 20:50:47.710161 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:47.710205 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:47.710371 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:47.712149 (+ 1778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:47.712160 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:47.712214 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:47.712681 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.712697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.714670 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.714676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.717243 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.717268 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.719230 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.719238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.719446 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.719507 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 20:50:47.719656 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 20:50:47.719874 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":27,"thread_start_us":306,"threads_started":1}
I20260926 20:50:47.720609 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.710903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.5.78.3:44325)
0926 20:50:47.712643 (+ 1740us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:47.712651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:47.712682 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:50:47.712779 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:47.712785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:47.713013 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:47.713859 (+ 846us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:47.714132 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.715467 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.718068 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.719508 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.719665 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.719905 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:47.720115 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 20:50:47.720283 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585,"thread_start_us":1446,"threads_started":1}
I20260926 20:50:47.721148 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.709833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.5.78.1:35465)
0926 20:50:47.711455 (+ 1622us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:47.711465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:47.711492 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:50:47.711602 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:47.711612 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:47.711875 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:47.712642 (+ 767us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:47.712878 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.714529 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.717460 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.718990 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.719127 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.719365 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:47.720617 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0926 20:50:47.720814 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1439,"thread_start_us":1277,"threads_started":1}
I20260926 20:50:47.721408 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.710620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:44325 (local address 127.0.0.1:54808)
0926 20:50:47.711224 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:47.711241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:47.711364 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:47.713240 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:47.713247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:47.713276 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:47.713674 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.713686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.715618 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.715623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.717879 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.717896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.719725 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.719731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.719908 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.719946 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:50:47.720072 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 20:50:47.720236 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":28,"thread_start_us":335,"threads_started":1}
I20260926 20:50:47.723603 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.711293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:33323 (local address 127.0.0.1:56608)
0926 20:50:47.711931 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:47.711951 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:47.712068 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:47.715174 (+ 3106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:47.715179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:47.715205 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:47.715683 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.715694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.717209 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.717215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.720682 (+ 3467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:47.720695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.722770 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.722776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:47.722953 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.722991 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:50:47.723078 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 20:50:47.723231 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":1229,"thread_start_us":597,"threads_started":1}
I20260926 20:50:47.724309 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:47.711730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.5.78.2:33323)
0926 20:50:47.714583 (+ 2853us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:47.714589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:47.714646 (+ 57us) server_negotiation.cc:407] Connection header received
0926 20:50:47.714739 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:47.714748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:47.714973 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:47.715164 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:47.715862 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.717030 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.720858 (+ 3828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:47.722272 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:47.722400 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:47.722624 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:47.723811 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0926 20:50:47.723977 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2718,"thread_start_us":2576,"threads_started":1}
I20260926 20:50:47.725090 22740 tablet_service.cc:1467] Processing CreateTablet for tablet d6b14c6f1d344c57a6820a75e01e2ff3 (DEFAULT_TABLE table=test-table [id=ba2d0689c44d4fca9bb85d0111e12e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:47.725055 23008 tablet_service.cc:1467] Processing CreateTablet for tablet d6b14c6f1d344c57a6820a75e01e2ff3 (DEFAULT_TABLE table=test-table [id=ba2d0689c44d4fca9bb85d0111e12e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:47.726354 22740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b14c6f1d344c57a6820a75e01e2ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:47.726500 23008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b14c6f1d344c57a6820a75e01e2ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:47.727816 22874 tablet_service.cc:1467] Processing CreateTablet for tablet d6b14c6f1d344c57a6820a75e01e2ff3 (DEFAULT_TABLE table=test-table [id=ba2d0689c44d4fca9bb85d0111e12e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:47.729357 22874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b14c6f1d344c57a6820a75e01e2ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:47.737838 23092 tablet_bootstrap.cc:492] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Bootstrap starting.
I20260926 20:50:47.740809 23093 tablet_bootstrap.cc:492] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Bootstrap starting.
I20260926 20:50:47.741747 23092 tablet_bootstrap.cc:654] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:47.743081 23094 tablet_bootstrap.cc:492] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Bootstrap starting.
I20260926 20:50:47.744131 23092 log.cc:824] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:47.745045 23093 tablet_bootstrap.cc:654] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:47.746048 23094 tablet_bootstrap.cc:654] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:47.746229 23093 log.cc:824] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:47.747229 23094 log.cc:824] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:47.747292 23092 tablet_bootstrap.cc:492] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: No bootstrap required, opened a new log
I20260926 20:50:47.747553 23092 ts_tablet_manager.cc:1397] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260926 20:50:47.749281 23093 tablet_bootstrap.cc:492] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: No bootstrap required, opened a new log
I20260926 20:50:47.749544 23093 ts_tablet_manager.cc:1397] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260926 20:50:47.749641 23094 tablet_bootstrap.cc:492] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: No bootstrap required, opened a new log
I20260926 20:50:47.749888 23094 ts_tablet_manager.cc:1397] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260926 20:50:47.756886 23094 raft_consensus.cc:348] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:47.757249 23094 raft_consensus.cc:374] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:47.757350 23094 raft_consensus.cc:729] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e1be06cbdc64d59bb26c58bcfac3b52, State: Initialized, Role: FOLLOWER
I20260926 20:50:47.757961 23094 consensus_queue.cc:260] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:47.759047 22939 heartbeater.cc:503] Master 127.5.78.62:46071 was elected leader, sending a full tablet report...
I20260926 20:50:47.758940 23092 raft_consensus.cc:348] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:47.759334 23092 raft_consensus.cc:374] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:47.759431 23092 raft_consensus.cc:729] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1149b63b50e84d529ec9264973d575ad, State: Initialized, Role: FOLLOWER
I20260926 20:50:47.759733 23094 ts_tablet_manager.cc:1428] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260926 20:50:47.759914 23093 raft_consensus.cc:348] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:47.760089 23092 consensus_queue.cc:260] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:47.760309 23093 raft_consensus.cc:374] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:47.760408 23093 raft_consensus.cc:729] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad048428eaa54a82b59d476e234ce9c6, State: Initialized, Role: FOLLOWER
I20260926 20:50:47.760985 23093 consensus_queue.cc:260] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:47.764535 23092 ts_tablet_manager.cc:1428] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Time spent starting tablet: real 0.017s user 0.006s sys 0.007s
I20260926 20:50:47.765511 23073 heartbeater.cc:503] Master 127.5.78.62:46071 was elected leader, sending a full tablet report...
I20260926 20:50:47.766230 23093 ts_tablet_manager.cc:1428] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
W20260926 20:50:47.788439 22806 tablet.cc:2369] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:47.848093 22940 tablet.cc:2369] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:47.882395 23074 tablet.cc:2369] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:48.039178 23099 raft_consensus.cc:482] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:50:48.039577 23099 raft_consensus.cc:504] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:48.041504 23099 leader_election.cc:288] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad048428eaa54a82b59d476e234ce9c6 (127.5.78.3:44325), 1e1be06cbdc64d59bb26c58bcfac3b52 (127.5.78.2:33323)
I20260926 20:50:48.049232 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.041991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:35921 (local address 127.5.78.3:44325)
0926 20:50:48.042208 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:48.042213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:48.042760 (+ 547us) server_negotiation.cc:407] Connection header received
0926 20:50:48.043045 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:48.043051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:48.043137 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:48.043252 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:48.044121 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.045245 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.047346 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.048182 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.048280 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.048359 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:48.048814 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 20:50:48.048970 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:50:48.049312 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.041877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:44325 (local address 127.5.78.1:35921)
0926 20:50:48.042625 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:48.042648 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:48.042796 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:48.043431 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:48.043436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:48.043472 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:48.043945 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:48.043958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.045403 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.045409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:48.047183 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:48.047195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.048404 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.048410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:48.048606 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.048681 (+ 75us) client_negotiation.cc:769] Sending connection context
0926 20:50:48.048758 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 20:50:48.048859 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":7,"thread_start_us":397,"threads_started":1}
I20260926 20:50:48.049508 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.042800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:33323 (local address 127.5.78.1:34953)
0926 20:50:48.043501 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:48.043521 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:48.043765 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:48.044174 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:48.044182 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:48.044210 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:48.044583 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:48.044594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.045808 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.045812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:48.047350 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:48.047363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.048614 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.048619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:48.048827 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.048859 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:50:48.048949 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 20:50:48.049089 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":37,"thread_start_us":341,"threads_started":1}
I20260926 20:50:48.049649 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.043093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:34953 (local address 127.5.78.2:33323)
0926 20:50:48.043329 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:48.043335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:48.043714 (+ 379us) server_negotiation.cc:407] Connection header received
0926 20:50:48.043944 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:48.043950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:48.044048 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:48.044141 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:48.044789 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.045651 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.047512 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.048398 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.048527 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.048620 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:48.049003 (+ 383us) server_negotiation.cc:299] Negotiation successful
0926 20:50:48.049149 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 20:50:48.050536 23028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b14c6f1d344c57a6820a75e01e2ff3" candidate_uuid: "1149b63b50e84d529ec9264973d575ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad048428eaa54a82b59d476e234ce9c6" is_pre_election: true
I20260926 20:50:48.050597 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b14c6f1d344c57a6820a75e01e2ff3" candidate_uuid: "1149b63b50e84d529ec9264973d575ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" is_pre_election: true
I20260926 20:50:48.051087 23028 raft_consensus.cc:2454] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1149b63b50e84d529ec9264973d575ad in term 0.
I20260926 20:50:48.051141 22894 raft_consensus.cc:2454] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1149b63b50e84d529ec9264973d575ad in term 0.
I20260926 20:50:48.051800 22694 leader_election.cc:302] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1149b63b50e84d529ec9264973d575ad, ad048428eaa54a82b59d476e234ce9c6; no voters:
I20260926 20:50:48.052202 23099 raft_consensus.cc:2781] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:50:48.052350 23099 raft_consensus.cc:482] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:50:48.052491 23099 raft_consensus.cc:3037] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:48.054744 23099 raft_consensus.cc:504] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:48.055347 23099 leader_election.cc:288] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [CANDIDATE]: Term 1 election: Requested vote from peers ad048428eaa54a82b59d476e234ce9c6 (127.5.78.3:44325), 1e1be06cbdc64d59bb26c58bcfac3b52 (127.5.78.2:33323)
I20260926 20:50:48.055797 23028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b14c6f1d344c57a6820a75e01e2ff3" candidate_uuid: "1149b63b50e84d529ec9264973d575ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad048428eaa54a82b59d476e234ce9c6"
I20260926 20:50:48.055835 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b14c6f1d344c57a6820a75e01e2ff3" candidate_uuid: "1149b63b50e84d529ec9264973d575ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52"
I20260926 20:50:48.055989 23028 raft_consensus.cc:3037] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:48.056030 22894 raft_consensus.cc:3037] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:48.058162 22894 raft_consensus.cc:2454] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1149b63b50e84d529ec9264973d575ad in term 1.
I20260926 20:50:48.058195 23028 raft_consensus.cc:2454] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1149b63b50e84d529ec9264973d575ad in term 1.
I20260926 20:50:48.058588 22694 leader_election.cc:302] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1149b63b50e84d529ec9264973d575ad, 1e1be06cbdc64d59bb26c58bcfac3b52; no voters:
I20260926 20:50:48.058844 23099 raft_consensus.cc:2781] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:48.059305 23099 raft_consensus.cc:686] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 1 LEADER]: Becoming Leader. State: Replica: 1149b63b50e84d529ec9264973d575ad, State: Running, Role: LEADER
I20260926 20:50:48.059773 23099 consensus_queue.cc:237] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } }
I20260926 20:50:48.064363 22616 catalog_manager.cc:5495] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 1149b63b50e84d529ec9264973d575ad (127.5.78.1). New cstate: current_term: 1 leader_uuid: "1149b63b50e84d529ec9264973d575ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1149b63b50e84d529ec9264973d575ad" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 35465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 } health_report { overall_health: UNKNOWN } } }
I20260926 20:50:48.089977 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.084496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.5.78.1:35465)
0926 20:50:48.084741 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:48.084747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:48.084859 (+ 112us) server_negotiation.cc:407] Connection header received
0926 20:50:48.085021 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:48.085026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:48.085108 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:48.085232 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:50:48.085551 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.086269 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.087774 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.088182 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.088285 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.088827 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:50:48.089336 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:50:48.089450 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:48.089601 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 20:50:48.089668 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:50:48.099169 22894 raft_consensus.cc:1261] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 1 FOLLOWER]: Refusing update from remote peer 1149b63b50e84d529ec9264973d575ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:48.099164 23028 raft_consensus.cc:1261] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 1 FOLLOWER]: Refusing update from remote peer 1149b63b50e84d529ec9264973d575ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:48.100191 23103 consensus_queue.cc:1035] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e1be06cbdc64d59bb26c58bcfac3b52" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:48.100596 23099 consensus_queue.cc:1035] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad048428eaa54a82b59d476e234ce9c6" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:48.129168 22615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:48.129423 22615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:48.133167 22615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9494382ccb44f4e93485faf6e73a6c7: Sending DeleteTablet for 3 replicas of tablet d6b14c6f1d344c57a6820a75e01e2ff3
I20260926 20:50:48.134454 22740 tablet_service.cc:1514] Processing DeleteTablet for tablet d6b14c6f1d344c57a6820a75e01e2ff3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:48 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:51154
I20260926 20:50:48.134734 22874 tablet_service.cc:1514] Processing DeleteTablet for tablet d6b14c6f1d344c57a6820a75e01e2ff3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:48 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:56608
I20260926 20:50:48.135625 23117 tablet_replica.cc:330] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: stopping tablet replica
I20260926 20:50:48.135710 23008 tablet_service.cc:1514] Processing DeleteTablet for tablet d6b14c6f1d344c57a6820a75e01e2ff3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:48 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:54808
I20260926 20:50:48.136184 23117 raft_consensus.cc:2229] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:48.137123 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22677
I20260926 20:50:48.137156 23118 tablet_replica.cc:330] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: stopping tablet replica
I20260926 20:50:48.137679 23118 raft_consensus.cc:2229] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:48.138199 23118 raft_consensus.cc:2258] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:48.139186 23116 tablet_replica.cc:330] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: stopping tablet replica
I20260926 20:50:48.139616 23116 raft_consensus.cc:2229] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:50:48.140283 23116 raft_consensus.cc:2258] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:48.142064 23117 raft_consensus.cc:2258] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:48.143097 23118 ts_tablet_manager.cc:1905] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:48.143690 23116 ts_tablet_manager.cc:1905] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:48.150259 23118 ts_tablet_manager.cc:1918] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:48.150487 23118 log.cc:1196] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/d6b14c6f1d344c57a6820a75e01e2ff3
I20260926 20:50:48.151054 23118 ts_tablet_manager.cc:1939] T d6b14c6f1d344c57a6820a75e01e2ff3 P ad048428eaa54a82b59d476e234ce9c6: Deleting consensus metadata
I20260926 20:50:48.152329 23116 ts_tablet_manager.cc:1918] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:48.152516 23116 log.cc:1196] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/d6b14c6f1d344c57a6820a75e01e2ff3
I20260926 20:50:48.152884 23117 ts_tablet_manager.cc:1905] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:48.153081 23116 ts_tablet_manager.cc:1939] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1149b63b50e84d529ec9264973d575ad: Deleting consensus metadata
I20260926 20:50:48.153131 22601 catalog_manager.cc:4841] TS ad048428eaa54a82b59d476e234ce9c6 (127.5.78.3:44325): tablet d6b14c6f1d344c57a6820a75e01e2ff3 (table test-table [id=ba2d0689c44d4fca9bb85d0111e12e8e]) successfully deleted
I20260926 20:50:48.155300 22601 catalog_manager.cc:4841] TS 1149b63b50e84d529ec9264973d575ad (127.5.78.1:35465): tablet d6b14c6f1d344c57a6820a75e01e2ff3 (table test-table [id=ba2d0689c44d4fca9bb85d0111e12e8e]) successfully deleted
I20260926 20:50:48.159444 23117 ts_tablet_manager.cc:1918] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:48.159646 23117 log.cc:1196] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/d6b14c6f1d344c57a6820a75e01e2ff3
I20260926 20:50:48.160233 23117 ts_tablet_manager.cc:1939] T d6b14c6f1d344c57a6820a75e01e2ff3 P 1e1be06cbdc64d59bb26c58bcfac3b52: Deleting consensus metadata
I20260926 20:50:48.162281 22601 catalog_manager.cc:4841] TS 1e1be06cbdc64d59bb26c58bcfac3b52 (127.5.78.2:33323): tablet d6b14c6f1d344c57a6820a75e01e2ff3 (table test-table [id=ba2d0689c44d4fca9bb85d0111e12e8e]) successfully deleted
Sep 26 20:50:48 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:50:48.171510 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.141377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.5.78.1:35465)
0926 20:50:48.141577 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:48.141585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:48.151399 (+ 9814us) server_negotiation.cc:407] Connection header received
0926 20:50:48.151610 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:48.151619 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:48.151712 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:48.151838 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:48.154458 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.155655 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.157577 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.158292 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.158473 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.163838 (+ 5365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:48.163853 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:48.163879 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:48.163923 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:48.168094 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:48.168983 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:48.168994 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:48.168998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:48.169181 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:48.169682 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:48.169688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:48.169692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:48.170167 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:48.170574 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:48.170872 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 20:50:48.171121 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 20:50:48.172477 22800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:48.271162 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22677
I20260926 20:50:48.289537 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22811
Sep 26 20:50:48 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:50:48.305115 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.290286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.5.78.2:33323)
0926 20:50:48.290534 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:48.290540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:48.290583 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:50:48.290753 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:48.290763 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:48.290828 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:48.290952 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:48.292223 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.293026 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.294453 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.294896 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.294995 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.299257 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:48.299271 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:48.299287 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:48.299320 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:48.302214 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:48.303013 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:48.303021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:48.303023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:48.303157 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:48.303570 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:48.303573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:48.303575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:48.303925 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:48.304240 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:48.304533 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 20:50:48.304764 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:50:48.305857 22934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:48.394640 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22811
I20260926 20:50:48.412220 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22945
Sep 26 20:50:48 dist-test-slave-z7j8 krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:50:48.428611 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:48.413251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.5.78.3:44325)
0926 20:50:48.413434 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:48.413439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:48.413459 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:50:48.413639 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:48.413643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:48.413722 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:48.413803 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:48.415256 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.416019 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.417400 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:48.417830 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:48.417938 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:48.422658 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:48.422677 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:48.422693 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:48.422737 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:48.425455 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:48.426187 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:48.426195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:48.426197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:48.426339 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:48.427038 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:48.427045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:48.427049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:48.427417 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:48.427730 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:48.428020 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 20:50:48.428244 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:50:48.429327 23068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:48.518366 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22945
I20260926 20:50:48.535068 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 22584
I20260926 20:50:48.536243 22646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:48.646674 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 22584
2026-09-26T20:50:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4526 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T20:50:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:50:48Z Disabled control of system clock
I20260926 20:50:48.723883 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33483
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:50:49.065192 23127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:49.065658 23127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:49.065822 23127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:49.075834 23127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:50:49.075960 23127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:49.075999 23127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:49.076049 23127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:50:49.076093 23127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:50:49.085693 23127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:49.087080 23127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:49.088822 23127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:49.102744 23133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:49.102777 23132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:49.103493 23135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:49.104000 23127 server_base.cc:1034] running on GCE node
I20260926 20:50:49.104879 23127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:49.106487 23127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:49.107748 23127 hybrid_clock.cc:648] HybridClock initialized: now 1790455849107697 us; error 36 us; skew 500 ppm
I20260926 20:50:49.108180 23127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:49.110730 23127 webserver.cc:460] Webserver started at http://127.5.78.62:37965/ using document root <none> and password file <none>
I20260926 20:50:49.111450 23127 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:49.111588 23127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:49.111918 23127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:49.114964 23127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "ee91e3990ceb47b086dcd81b3be9d31d"
format_stamp: "Formatted at 2026-09-26 20:50:49 on dist-test-slave-z7j8"
server_key: "fd213cfe2388171697789598518fcea3"
server_key_iv: "5d71ff3d95f4b254881eca2e7fa4c211"
server_key_version: "encryptionkey@0"
I20260926 20:50:49.115768 23127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "ee91e3990ceb47b086dcd81b3be9d31d"
format_stamp: "Formatted at 2026-09-26 20:50:49 on dist-test-slave-z7j8"
server_key: "fd213cfe2388171697789598518fcea3"
server_key_iv: "5d71ff3d95f4b254881eca2e7fa4c211"
server_key_version: "encryptionkey@0"
I20260926 20:50:49.121063 23127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:50:49.125401 23141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:49.126935 23127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:50:49.127133 23127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ee91e3990ceb47b086dcd81b3be9d31d"
format_stamp: "Formatted at 2026-09-26 20:50:49 on dist-test-slave-z7j8"
server_key: "fd213cfe2388171697789598518fcea3"
server_key_iv: "5d71ff3d95f4b254881eca2e7fa4c211"
server_key_version: "encryptionkey@0"
I20260926 20:50:49.127375 23127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:49.140826 23127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:49.142089 23127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:49.142474 23127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:49.164039 23127 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33483
I20260926 20:50:49.164053 23192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33483 every 8 connection(s)
I20260926 20:50:49.165427 23127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:50:49.168473 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23127
I20260926 20:50:49.168669 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:50:49.169229 5432 external_mini_cluster.cc:1428] Setting key d70b16d409a23d3cbd52bfb27ba5e489
I20260926 20:50:49.169761 23193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:49.183444 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:49.171217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.5.78.62:33483)
0926 20:50:49.172235 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:49.172246 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:49.172287 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:50:49.174344 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:49.174367 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:49.174919 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:49.175264 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:49.175893 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:49.177231 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:49.179040 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:49.179710 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:49.179842 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:49.181135 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:49.181142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:49.181154 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:49.181713 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:49.181785 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:49.181804 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:49.181888 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:49.182130 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 20:50:49.182513 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":405,"threads_started":1}
I20260926 20:50:49.183539 23193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d: Bootstrap starting.
I20260926 20:50:49.187065 23193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:49.188050 23193 log.cc:824] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:49.190671 23193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d: No bootstrap required, opened a new log
I20260926 20:50:49.198686 23193 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33483 } }
I20260926 20:50:49.199074 23193 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:49.199205 23193 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee91e3990ceb47b086dcd81b3be9d31d, State: Initialized, Role: FOLLOWER
I20260926 20:50:49.199836 23193 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33483 } }
I20260926 20:50:49.200078 23193 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:50:49.200228 23193 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:50:49.200416 23193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:49.202786 23193 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33483 } }
I20260926 20:50:49.203338 23193 leader_election.cc:302] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee91e3990ceb47b086dcd81b3be9d31d; no voters:
I20260926 20:50:49.204062 23193 leader_election.cc:288] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:50:49.204192 23198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:49.205032 23198 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [term 1 LEADER]: Becoming Leader. State: Replica: ee91e3990ceb47b086dcd81b3be9d31d, State: Running, Role: LEADER
I20260926 20:50:49.205596 23198 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33483 } }
I20260926 20:50:49.206508 23193 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:50:49.208586 23200 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee91e3990ceb47b086dcd81b3be9d31d. Latest consensus state: current_term: 1 leader_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33483 } } }
I20260926 20:50:49.208706 23199 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee91e3990ceb47b086dcd81b3be9d31d" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33483 } } }
I20260926 20:50:49.209161 23199 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:49.209184 23200 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:49.211979 23203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:50:49.217108 23203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:50:49.226187 23203 catalog_manager.cc:1347] Generated new cluster ID: 0ba82e9bca854ad380b16e02bae465c2
I20260926 20:50:49.226305 23203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:50:49.249683 23203 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:50:49.251142 23203 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:50:49.264873 23203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d: Generated new TSK 0
I20260926 20:50:49.265769 23203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:50:49.285658 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33483
--builtin_ntp_servers=127.5.78.20:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:50:49.644109 23217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:49.644413 23217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:49.644538 23217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:49.651979 23217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:49.652107 23217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:49.652284 23217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:50:49.661624 23217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:49.662997 23217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:49.664896 23217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:49.678542 23222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:49.678509 23225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:49.678604 23223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:49.679337 23217 server_base.cc:1034] running on GCE node
I20260926 20:50:49.679999 23217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:49.680984 23217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:49.682193 23217 hybrid_clock.cc:648] HybridClock initialized: now 1790455849682159 us; error 31 us; skew 500 ppm
I20260926 20:50:49.682592 23217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:49.685056 23217 webserver.cc:460] Webserver started at http://127.5.78.1:42719/ using document root <none> and password file <none>
I20260926 20:50:49.685784 23217 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:49.685917 23217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:49.686240 23217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:49.689147 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "91fed739af014aa1ad9a652b41af8668"
format_stamp: "Formatted at 2026-09-26 20:50:49 on dist-test-slave-z7j8"
server_key: "17b1450d65ae3946c2a3f86dd9631c8a"
server_key_iv: "7701a28e8aeef933c7cd042dd81b13c3"
server_key_version: "encryptionkey@0"
I20260926 20:50:49.690040 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "91fed739af014aa1ad9a652b41af8668"
format_stamp: "Formatted at 2026-09-26 20:50:49 on dist-test-slave-z7j8"
server_key: "17b1450d65ae3946c2a3f86dd9631c8a"
server_key_iv: "7701a28e8aeef933c7cd042dd81b13c3"
server_key_version: "encryptionkey@0"
I20260926 20:50:49.695211 23217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:50:49.699716 23231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:49.701256 23217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260926 20:50:49.701455 23217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "91fed739af014aa1ad9a652b41af8668"
format_stamp: "Formatted at 2026-09-26 20:50:49 on dist-test-slave-z7j8"
server_key: "17b1450d65ae3946c2a3f86dd9631c8a"
server_key_iv: "7701a28e8aeef933c7cd042dd81b13c3"
server_key_version: "encryptionkey@0"
I20260926 20:50:49.701725 23217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:49.751961 23217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:49.753001 23217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:49.753341 23217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:49.754505 23217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:49.756225 23217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:49.756383 23217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:49.756541 23217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:49.756647 23217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:49.793177 23217 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:36379
I20260926 20:50:49.793200 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:36379 every 8 connection(s)
I20260926 20:50:49.794777 23217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:50:49.797009 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23217
I20260926 20:50:49.797148 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:50:49.797742 5432 external_mini_cluster.cc:1428] Setting key 3d9b6f274f84136ce889d247f34936a0
I20260926 20:50:49.801179 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33483
--builtin_ntp_servers=127.5.78.20:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:50:49.811692 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:49.797635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:48257 (local address 127.5.78.62:33483)
0926 20:50:49.798220 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:49.798227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:49.799629 (+ 1402us) server_negotiation.cc:407] Connection header received
0926 20:50:49.802546 (+ 2917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:49.802553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:49.802631 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:49.802750 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:49.804358 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:49.805343 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:49.808256 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:49.808718 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:49.808852 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:49.810720 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:49.810725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:49.810728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:49.810751 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:49.810805 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:49.810818 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:49.810922 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:49.811190 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 20:50:49.811308 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":309,"threads_started":1}
I20260926 20:50:49.812940 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:49.798471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33483 (local address 127.5.78.1:48257)
0926 20:50:49.799475 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:49.799511 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:49.802129 (+ 2618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:49.803057 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:49.803071 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:49.803206 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:49.804129 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:49.804161 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:49.805538 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:49.805546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:49.807995 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:49.808014 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:49.808990 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:49.809000 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:49.809374 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:49.810269 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:50:49.810272 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:49.810318 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:50:49.810471 (+ 153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:49.810924 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:49.810939 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:49.810950 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:50:49.811161 (+ 211us) client_negotiation.cc:240] Negotiation successful
0926 20:50:49.811677 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":325,"threads_started":1}
I20260926 20:50:49.814953 23344 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33483
I20260926 20:50:49.815328 23344 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:49.816169 23344 heartbeater.cc:511] Master 127.5.78.62:33483 requested a full tablet report, sending...
I20260926 20:50:49.818653 23158 ts_manager.cc:194] Registered new tserver with Master: 91fed739af014aa1ad9a652b41af8668 (127.5.78.1:36379)
I20260926 20:50:49.820941 23158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.1:48257
W20260926 20:50:50.143392 23348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:50.143675 23348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:50.143760 23348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:50.152227 23348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:50.152330 23348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:50.152437 23348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:50:50.161883 23348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:50.163187 23348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:50.165005 23348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:50.179941 23353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:50.179988 23354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:50.180034 23356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:50.180962 23348 server_base.cc:1034] running on GCE node
I20260926 20:50:50.181591 23348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:50.182662 23348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:50.183892 23348 hybrid_clock.cc:648] HybridClock initialized: now 1790455850183845 us; error 61 us; skew 500 ppm
I20260926 20:50:50.184219 23348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:50.186520 23348 webserver.cc:460] Webserver started at http://127.5.78.2:40741/ using document root <none> and password file <none>
I20260926 20:50:50.187209 23348 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:50.187336 23348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:50.187647 23348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:50.190486 23348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "902e4f172eb14721b616e3a9c34b7de3"
format_stamp: "Formatted at 2026-09-26 20:50:50 on dist-test-slave-z7j8"
server_key: "53203c328906d1de5cdf9c6f72c52697"
server_key_iv: "ee9ff5b9df7f5c75471a2b1e48706bf5"
server_key_version: "encryptionkey@0"
I20260926 20:50:50.191267 23348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "902e4f172eb14721b616e3a9c34b7de3"
format_stamp: "Formatted at 2026-09-26 20:50:50 on dist-test-slave-z7j8"
server_key: "53203c328906d1de5cdf9c6f72c52697"
server_key_iv: "ee9ff5b9df7f5c75471a2b1e48706bf5"
server_key_version: "encryptionkey@0"
I20260926 20:50:50.196138 23348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:50:50.199481 23362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:50.200806 23348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:50:50.201004 23348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "902e4f172eb14721b616e3a9c34b7de3"
format_stamp: "Formatted at 2026-09-26 20:50:50 on dist-test-slave-z7j8"
server_key: "53203c328906d1de5cdf9c6f72c52697"
server_key_iv: "ee9ff5b9df7f5c75471a2b1e48706bf5"
server_key_version: "encryptionkey@0"
I20260926 20:50:50.201228 23348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:50.222991 23348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:50.223831 23348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:50.224113 23348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:50.225063 23348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:50.226516 23348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:50.226644 23348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:50.226765 23348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:50.226850 23348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:50.259701 23348 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:32805
I20260926 20:50:50.259796 23474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:32805 every 8 connection(s)
I20260926 20:50:50.260998 23348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:50:50.270995 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23348
I20260926 20:50:50.271198 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:50:50.271716 5432 external_mini_cluster.cc:1428] Setting key 790a1618a32cfbf476f5b64558ef0cbd
I20260926 20:50:50.274595 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.263830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:34483 (local address 127.5.78.62:33483)
0926 20:50:50.264062 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:50.264069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:50.265470 (+ 1401us) server_negotiation.cc:407] Connection header received
0926 20:50:50.267341 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:50.267346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:50.267412 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:50.267503 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:50.268655 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.269383 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.271495 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.271975 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.272093 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.273732 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:50.273738 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:50.273743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:50.273765 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:50.273802 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:50.273819 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:50.273915 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:50.274225 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 20:50:50.274305 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 20:50:50.275259 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33483
--builtin_ntp_servers=127.5.78.20:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:50:50.276235 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.264365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33483 (local address 127.5.78.2:34483)
0926 20:50:50.265296 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:50.265334 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:50.267089 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:50.267740 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:50.267748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:50.267844 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:50.268411 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.268428 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.269559 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.269564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.271317 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.271334 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.272215 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.272223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.272518 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.273336 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:50:50.273340 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:50.273390 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:50:50.273516 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:50.273959 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:50.273974 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:50.273986 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:50:50.274225 (+ 239us) client_negotiation.cc:240] Negotiation successful
0926 20:50:50.274742 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":306,"threads_started":1}
I20260926 20:50:50.277683 23475 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33483
I20260926 20:50:50.278033 23475 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:50.279007 23475 heartbeater.cc:511] Master 127.5.78.62:33483 requested a full tablet report, sending...
I20260926 20:50:50.280684 23158 ts_manager.cc:194] Registered new tserver with Master: 902e4f172eb14721b616e3a9c34b7de3 (127.5.78.2:32805)
I20260926 20:50:50.281489 23158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.2:34483
W20260926 20:50:50.614051 23478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:50.614328 23478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:50.614508 23478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:50.622327 23478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:50.622483 23478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:50.622670 23478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:50:50.631994 23478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:33483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:50.633293 23478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:50.635250 23478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:50.647734 23483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:50.648258 23484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:50.649288 23486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:50.650580 23478 server_base.cc:1034] running on GCE node
I20260926 20:50:50.651255 23478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:50.652334 23478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:50.653596 23478 hybrid_clock.cc:648] HybridClock initialized: now 1790455850653558 us; error 54 us; skew 500 ppm
I20260926 20:50:50.653936 23478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:50.656270 23478 webserver.cc:460] Webserver started at http://127.5.78.3:45593/ using document root <none> and password file <none>
I20260926 20:50:50.656963 23478 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:50.657100 23478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:50.657439 23478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:50.660485 23478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "950bf59cd5ab42c382347cae4e815b87"
format_stamp: "Formatted at 2026-09-26 20:50:50 on dist-test-slave-z7j8"
server_key: "8a401a03e6896c28dde63a13fa879f8b"
server_key_iv: "c09a2d1a56db476bf108cd7876b66212"
server_key_version: "encryptionkey@0"
I20260926 20:50:50.661274 23478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "950bf59cd5ab42c382347cae4e815b87"
format_stamp: "Formatted at 2026-09-26 20:50:50 on dist-test-slave-z7j8"
server_key: "8a401a03e6896c28dde63a13fa879f8b"
server_key_iv: "c09a2d1a56db476bf108cd7876b66212"
server_key_version: "encryptionkey@0"
I20260926 20:50:50.666321 23478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:50:50.669883 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:50.671602 23478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 20:50:50.671787 23478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "950bf59cd5ab42c382347cae4e815b87"
format_stamp: "Formatted at 2026-09-26 20:50:50 on dist-test-slave-z7j8"
server_key: "8a401a03e6896c28dde63a13fa879f8b"
server_key_iv: "c09a2d1a56db476bf108cd7876b66212"
server_key_version: "encryptionkey@0"
I20260926 20:50:50.672019 23478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:50.687482 23478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:50.688373 23478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:50.688678 23478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:50.689743 23478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:50.691455 23478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:50.691574 23478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:50.691717 23478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:50.691807 23478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:50.726068 23478 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:35083
I20260926 20:50:50.726122 23604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:35083 every 8 connection(s)
I20260926 20:50:50.727438 23478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:50:50.731746 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23478
I20260926 20:50:50.731916 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:50:50.732388 5432 external_mini_cluster.cc:1428] Setting key a06a3029cca34602f7cc1039d0adb5a1
I20260926 20:50:50.741902 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.730142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:40103 (local address 127.5.78.62:33483)
0926 20:50:50.730330 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:50.730334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:50.732139 (+ 1805us) server_negotiation.cc:407] Connection header received
0926 20:50:50.734614 (+ 2475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:50.734621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:50.734725 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:50.734848 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:50.736280 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.737077 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.738859 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.739290 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.739446 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.740951 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:50.740956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:50.740960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:50.740979 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:50.741012 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:50.741025 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:50.741124 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:50.741400 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 20:50:50.741508 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:50:50.742990 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.730759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33483 (local address 127.5.78.3:40103)
0926 20:50:50.731904 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:50.731988 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:50.734233 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:50.735119 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:50.735129 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:50.735228 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:50.736092 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.736115 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.737242 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.737249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.738695 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.738708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.739540 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.739548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.739805 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.740578 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:50:50.740580 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:50.740632 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:50:50.740719 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:50.741161 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:50.741172 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:50.741179 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:50:50.741386 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 20:50:50.741826 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":350,"threads_started":1}
I20260926 20:50:50.744449 23605 heartbeater.cc:347] Connected to a master server at 127.5.78.62:33483
I20260926 20:50:50.744829 23605 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:50.745718 23605 heartbeater.cc:511] Master 127.5.78.62:33483 requested a full tablet report, sending...
I20260926 20:50:50.747248 23158 ts_manager.cc:194] Registered new tserver with Master: 950bf59cd5ab42c382347cae4e815b87 (127.5.78.3:35083)
I20260926 20:50:50.748282 23158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.3:40103
I20260926 20:50:50.748374 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:50:50.759460 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.753614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.5.78.62:33483)
0926 20:50:50.753808 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:50.753812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:50.754327 (+ 515us) server_negotiation.cc:407] Connection header received
0926 20:50:50.754531 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:50.754535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:50.754614 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:50.754701 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:50.755195 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.755895 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.757323 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.757737 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.757830 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.758877 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:50.758882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:50.758885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:50.758902 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:50.758945 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:50.758957 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:50.759035 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:50.759161 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 20:50:50.759236 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:50:50.763437 23158 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:50:50.770344 23158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:50:50.807595 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.783693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:36379 (local address 127.0.0.1:33562)
0926 20:50:50.794882 (+ 11189us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:50.794963 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:50.795224 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:50.798480 (+ 3256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:50.798489 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:50.798521 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:50.798939 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.798957 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.801229 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.801235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.803862 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.803879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.806531 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.806538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.806815 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.806876 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 20:50:50.807005 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 20:50:50.807202 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10800,"threads_started":1}
I20260926 20:50:50.809190 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.783800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33562 (local address 127.5.78.1:36379)
0926 20:50:50.797385 (+ 13585us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:50.797396 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:50.797431 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:50:50.797565 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:50.797574 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:50.797878 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:50.798118 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:50.799199 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.801030 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.804136 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.805900 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.806071 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.806372 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:50.808529 (+ 2157us) server_negotiation.cc:299] Negotiation successful
0926 20:50:50.808709 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13406,"thread_start_us":13710,"threads_started":1}
I20260926 20:50:50.809832 23617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.783836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:32805 (local address 127.0.0.1:54426)
0926 20:50:50.798121 (+ 14285us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:50.798148 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:50.798280 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:50.799316 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:50.799324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:50.799353 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:50.799805 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.799821 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.802957 (+ 3136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.802963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.805402 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.805419 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.808968 (+ 3549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.808976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.809178 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.809224 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 20:50:50.809312 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 20:50:50.809461 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14121,"thread_start_us":10417,"threads_started":1}
I20260926 20:50:50.811051 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.784551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.5.78.2:32805)
0926 20:50:50.797508 (+ 12957us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:50.797518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:50.798439 (+ 921us) server_negotiation.cc:407] Connection header received
0926 20:50:50.798553 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:50.798563 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:50.798837 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:50.799050 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:50.800018 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.802794 (+ 2776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.805945 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.808742 (+ 2797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.809729 (+ 987us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.810128 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:50.810226 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 20:50:50.810496 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12340,"thread_start_us":12704,"threads_started":1}
I20260926 20:50:50.814122 23279 tablet_service.cc:1467] Processing CreateTablet for tablet 01134b7da20a4915a8a8b9e418646cc3 (DEFAULT_TABLE table=test-table [id=ea9f53f889ce4d4ba5fd3bb6680dd91c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:50.815685 23279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01134b7da20a4915a8a8b9e418646cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:50.815454 23410 tablet_service.cc:1467] Processing CreateTablet for tablet 01134b7da20a4915a8a8b9e418646cc3 (DEFAULT_TABLE table=test-table [id=ea9f53f889ce4d4ba5fd3bb6680dd91c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:50.816090 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.795375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:35083 (local address 127.0.0.1:59462)
0926 20:50:50.808206 (+ 12831us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:50.808227 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:50.808339 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:50.809182 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:50.809186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:50.809213 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:50.809377 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.809388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.811218 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.811224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.813018 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:50.813032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.815257 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.815264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:50.815469 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.815512 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 20:50:50.815627 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 20:50:50.815758 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12703,"thread_start_us":16503,"threads_started":1}
I20260926 20:50:50.816206 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:50.802747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59462 (local address 127.5.78.3:35083)
0926 20:50:50.804128 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:50.804139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:50.808302 (+ 4163us) server_negotiation.cc:407] Connection header received
0926 20:50:50.808491 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:50.808498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:50.808722 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:50.808902 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:50.809563 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.811031 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.813263 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:50.814724 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:50.814870 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:50.815149 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:50.815686 (+ 537us) server_negotiation.cc:299] Negotiation successful
0926 20:50:50.815850 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1201,"thread_start_us":2117,"threads_started":1}
I20260926 20:50:50.816807 23410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01134b7da20a4915a8a8b9e418646cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:50.819648 23540 tablet_service.cc:1467] Processing CreateTablet for tablet 01134b7da20a4915a8a8b9e418646cc3 (DEFAULT_TABLE table=test-table [id=ea9f53f889ce4d4ba5fd3bb6680dd91c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:50.821060 23540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01134b7da20a4915a8a8b9e418646cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:50.824388 23344 heartbeater.cc:503] Master 127.5.78.62:33483 was elected leader, sending a full tablet report...
I20260926 20:50:50.828642 23623 tablet_bootstrap.cc:492] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Bootstrap starting.
I20260926 20:50:50.831568 23624 tablet_bootstrap.cc:492] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Bootstrap starting.
I20260926 20:50:50.832196 23623 tablet_bootstrap.cc:654] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:50.833520 23623 log.cc:824] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:50.835300 23626 tablet_bootstrap.cc:492] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Bootstrap starting.
I20260926 20:50:50.835412 23624 tablet_bootstrap.cc:654] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:50.837095 23624 log.cc:824] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:50.838688 23626 tablet_bootstrap.cc:654] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:50.839680 23626 log.cc:824] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:50.840849 23624 tablet_bootstrap.cc:492] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: No bootstrap required, opened a new log
I20260926 20:50:50.841122 23624 ts_tablet_manager.cc:1397] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 20:50:50.842872 23623 tablet_bootstrap.cc:492] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: No bootstrap required, opened a new log
I20260926 20:50:50.842934 23626 tablet_bootstrap.cc:492] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: No bootstrap required, opened a new log
I20260926 20:50:50.843189 23623 ts_tablet_manager.cc:1397] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260926 20:50:50.843205 23626 ts_tablet_manager.cc:1397] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 20:50:50.851173 23623 raft_consensus.cc:348] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:50.851624 23623 raft_consensus.cc:374] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:50.851759 23623 raft_consensus.cc:729] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91fed739af014aa1ad9a652b41af8668, State: Initialized, Role: FOLLOWER
I20260926 20:50:50.851814 23624 raft_consensus.cc:348] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:50.852325 23624 raft_consensus.cc:374] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:50.852427 23624 raft_consensus.cc:729] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 902e4f172eb14721b616e3a9c34b7de3, State: Initialized, Role: FOLLOWER
I20260926 20:50:50.852447 23623 consensus_queue.cc:260] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:50.853031 23624 consensus_queue.cc:260] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:50.854693 23623 ts_tablet_manager.cc:1428] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260926 20:50:50.855351 23626 raft_consensus.cc:348] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:50.855840 23475 heartbeater.cc:503] Master 127.5.78.62:33483 was elected leader, sending a full tablet report...
I20260926 20:50:50.855844 23626 raft_consensus.cc:374] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:50.855957 23626 raft_consensus.cc:729] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950bf59cd5ab42c382347cae4e815b87, State: Initialized, Role: FOLLOWER
I20260926 20:50:50.856508 23624 ts_tablet_manager.cc:1428] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260926 20:50:50.856575 23626 consensus_queue.cc:260] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:50.857846 23605 heartbeater.cc:503] Master 127.5.78.62:33483 was elected leader, sending a full tablet report...
I20260926 20:50:50.858565 23626 ts_tablet_manager.cc:1428] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260926 20:50:50.979204 23606 tablet.cc:2369] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:51.013109 23476 tablet.cc:2369] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:51.047504 23345 tablet.cc:2369] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:51.068950 23630 raft_consensus.cc:482] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:50:51.069316 23630 raft_consensus.cc:504] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:51.071135 23630 leader_election.cc:288] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91fed739af014aa1ad9a652b41af8668 (127.5.78.1:36379), 950bf59cd5ab42c382347cae4e815b87 (127.5.78.3:35083)
I20260926 20:50:51.073129 23629 raft_consensus.cc:482] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:50:51.073457 23629 raft_consensus.cc:504] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:51.075548 23629 leader_election.cc:288] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 902e4f172eb14721b616e3a9c34b7de3 (127.5.78.2:32805), 950bf59cd5ab42c382347cae4e815b87 (127.5.78.3:35083)
I20260926 20:50:51.080533 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.071488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:37085 (local address 127.5.78.1:36379)
0926 20:50:51.071774 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.071783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.072372 (+ 589us) server_negotiation.cc:407] Connection header received
0926 20:50:51.072611 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.072617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.072703 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.072803 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:51.073383 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.074342 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.077147 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.078717 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.078814 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.078909 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.079969 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.080170 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:50:51.080710 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.071509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:35083 (local address 127.5.78.2:41235)
0926 20:50:51.072451 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:51.072464 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:51.072573 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:51.073269 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:51.073274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:51.073298 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:51.073752 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.073767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.075162 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.075167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.078296 (+ 3129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.078315 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.079921 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.079927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.080096 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.080138 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:50:51.080225 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 20:50:51.080358 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":303,"thread_start_us":404,"threads_started":1}
I20260926 20:50:51.081382 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.071897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:41235 (local address 127.5.78.3:35083)
0926 20:50:51.072127 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.072134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.072958 (+ 824us) server_negotiation.cc:407] Connection header received
0926 20:50:51.073043 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.073048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.073134 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.073228 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:51.073930 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.075012 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.078509 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.079735 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.079871 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.080008 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.080978 (+ 970us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.081138 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:50:51.082099 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.071509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:36379 (local address 127.5.78.2:37085)
0926 20:50:51.072253 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:51.072268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:51.072462 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:51.072838 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:51.072843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:51.072866 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:51.073218 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.073230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.074535 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.074541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.076873 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.076887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.079540 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.079546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.079751 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.079800 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 20:50:51.081400 (+ 1600us) client_negotiation.cc:240] Negotiation successful
0926 20:50:51.081606 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":30,"thread_start_us":404,"threads_started":1}
I20260926 20:50:51.082504 23560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01134b7da20a4915a8a8b9e418646cc3" candidate_uuid: "902e4f172eb14721b616e3a9c34b7de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950bf59cd5ab42c382347cae4e815b87" is_pre_election: true
I20260926 20:50:51.083074 23560 raft_consensus.cc:2454] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 902e4f172eb14721b616e3a9c34b7de3 in term 0.
I20260926 20:50:51.083559 23299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01134b7da20a4915a8a8b9e418646cc3" candidate_uuid: "902e4f172eb14721b616e3a9c34b7de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91fed739af014aa1ad9a652b41af8668" is_pre_election: true
I20260926 20:50:51.083999 23363 leader_election.cc:302] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 902e4f172eb14721b616e3a9c34b7de3, 950bf59cd5ab42c382347cae4e815b87; no voters:
I20260926 20:50:51.084327 23299 raft_consensus.cc:2454] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 902e4f172eb14721b616e3a9c34b7de3 in term 0.
I20260926 20:50:51.084472 23630 raft_consensus.cc:2781] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:50:51.084615 23630 raft_consensus.cc:482] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:50:51.084720 23630 raft_consensus.cc:3037] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:51.085810 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.075866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:32805 (local address 127.5.78.1:44851)
0926 20:50:51.076490 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:51.076508 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:51.076688 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:51.078848 (+ 2160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:51.078855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:51.078885 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:51.079260 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.079269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.080833 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.080838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.083093 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.083112 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.085003 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.085011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.085200 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.085246 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 20:50:51.085347 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 20:50:51.085485 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":634,"threads_started":1}
I20260926 20:50:51.085973 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.076323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:44851 (local address 127.5.78.2:32805)
0926 20:50:51.076493 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.076498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.076581 (+ 83us) server_negotiation.cc:407] Connection header received
0926 20:50:51.076827 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.076832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.076925 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.076991 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:51.079561 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.080661 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.083360 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.084556 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.084678 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.084828 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.085409 (+ 581us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.085607 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:50:51.087790 23630 raft_consensus.cc:504] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:51.088894 23560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01134b7da20a4915a8a8b9e418646cc3" candidate_uuid: "902e4f172eb14721b616e3a9c34b7de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950bf59cd5ab42c382347cae4e815b87"
I20260926 20:50:51.088943 23299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01134b7da20a4915a8a8b9e418646cc3" candidate_uuid: "902e4f172eb14721b616e3a9c34b7de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91fed739af014aa1ad9a652b41af8668"
I20260926 20:50:51.089121 23560 raft_consensus.cc:3037] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:51.089239 23299 raft_consensus.cc:3037] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:51.091142 23430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01134b7da20a4915a8a8b9e418646cc3" candidate_uuid: "91fed739af014aa1ad9a652b41af8668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "902e4f172eb14721b616e3a9c34b7de3" is_pre_election: true
I20260926 20:50:51.091651 23430 raft_consensus.cc:2379] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91fed739af014aa1ad9a652b41af8668 in current term 1: Already voted for candidate 902e4f172eb14721b616e3a9c34b7de3 in this term.
I20260926 20:50:51.091630 23560 raft_consensus.cc:2454] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 902e4f172eb14721b616e3a9c34b7de3 in term 1.
I20260926 20:50:51.092077 23630 leader_election.cc:288] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [CANDIDATE]: Term 1 election: Requested vote from peers 91fed739af014aa1ad9a652b41af8668 (127.5.78.1:36379), 950bf59cd5ab42c382347cae4e815b87 (127.5.78.3:35083)
I20260926 20:50:51.092133 23363 leader_election.cc:302] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 902e4f172eb14721b616e3a9c34b7de3, 950bf59cd5ab42c382347cae4e815b87; no voters:
I20260926 20:50:51.092398 23630 raft_consensus.cc:2781] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:51.092604 23299 raft_consensus.cc:2454] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 902e4f172eb14721b616e3a9c34b7de3 in term 1.
I20260926 20:50:51.092947 23630 raft_consensus.cc:686] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 1 LEADER]: Becoming Leader. State: Replica: 902e4f172eb14721b616e3a9c34b7de3, State: Running, Role: LEADER
I20260926 20:50:51.093567 23630 consensus_queue.cc:237] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } }
I20260926 20:50:51.096478 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.076961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:35083 (local address 127.5.78.1:50243)
0926 20:50:51.086118 (+ 9157us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:51.086138 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:51.086299 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:51.086812 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:51.086818 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:51.086847 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:51.087028 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.087038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.093164 (+ 6126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.093170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.094491 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:51.094509 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.095679 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.095685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:51.095887 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.095912 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:50:51.096023 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 20:50:51.096167 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9038,"thread_start_us":13187,"threads_started":1}
I20260926 20:50:51.096535 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.077399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:50243 (local address 127.5.78.3:35083)
0926 20:50:51.079063 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.079069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.086205 (+ 7136us) server_negotiation.cc:407] Connection header received
0926 20:50:51.086498 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.086507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.086619 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.086749 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:51.087222 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.092979 (+ 5757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.094698 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.095507 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.095664 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.095770 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.096051 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.096192 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1524,"thread_start_us":1311,"threads_started":1}
I20260926 20:50:51.097419 23560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01134b7da20a4915a8a8b9e418646cc3" candidate_uuid: "91fed739af014aa1ad9a652b41af8668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950bf59cd5ab42c382347cae4e815b87" is_pre_election: true
I20260926 20:50:51.097720 23560 raft_consensus.cc:2379] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91fed739af014aa1ad9a652b41af8668 in current term 1: Already voted for candidate 902e4f172eb14721b616e3a9c34b7de3 in this term.
I20260926 20:50:51.098452 23232 leader_election.cc:302] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91fed739af014aa1ad9a652b41af8668; no voters: 902e4f172eb14721b616e3a9c34b7de3, 950bf59cd5ab42c382347cae4e815b87
I20260926 20:50:51.098945 23629 raft_consensus.cc:2726] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 20:50:51.098771 23156 catalog_manager.cc:5495] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 902e4f172eb14721b616e3a9c34b7de3 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "902e4f172eb14721b616e3a9c34b7de3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "902e4f172eb14721b616e3a9c34b7de3" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 32805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 } health_report { overall_health: UNKNOWN } } }
I20260926 20:50:51.164263 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.158572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54440 (local address 127.5.78.2:32805)
0926 20:50:51.158788 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.158793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.158811 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:50:51.158975 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.158979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.159073 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.159200 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:50:51.159602 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.160342 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.161851 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.162265 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.162361 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.162961 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:50:51.163595 (+ 634us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:50:51.163737 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.163864 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.163957 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:50:51.173565 23560 raft_consensus.cc:1261] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 1 FOLLOWER]: Refusing update from remote peer 902e4f172eb14721b616e3a9c34b7de3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:51.173619 23299 raft_consensus.cc:1261] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 1 FOLLOWER]: Refusing update from remote peer 902e4f172eb14721b616e3a9c34b7de3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:51.174626 23630 consensus_queue.cc:1035] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950bf59cd5ab42c382347cae4e815b87" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 35083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:51.174927 23637 consensus_queue.cc:1035] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91fed739af014aa1ad9a652b41af8668" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 36379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:51.186488 23641 mvcc.cc:205] Tried to move back new op lower bound from 7333707166397026304 to 7333707166083579904. Current Snapshot: MvccSnapshot[applied={T|T < 7333707166397026304}]
I20260926 20:50:51.187042 23640 mvcc.cc:205] Tried to move back new op lower bound from 7333707166397026304 to 7333707166083579904. Current Snapshot: MvccSnapshot[applied={T|T < 7333707166397026304}]
I20260926 20:50:51.202595 23156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:51.203018 23156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:51.207226 23156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee91e3990ceb47b086dcd81b3be9d31d: Sending DeleteTablet for 3 replicas of tablet 01134b7da20a4915a8a8b9e418646cc3
I20260926 20:50:51.208463 23279 tablet_service.cc:1514] Processing DeleteTablet for tablet 01134b7da20a4915a8a8b9e418646cc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:51 UTC) from {username='slave'} at 127.0.0.1:33562
I20260926 20:50:51.208635 23540 tablet_service.cc:1514] Processing DeleteTablet for tablet 01134b7da20a4915a8a8b9e418646cc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:51 UTC) from {username='slave'} at 127.0.0.1:59462
I20260926 20:50:51.209522 23410 tablet_service.cc:1514] Processing DeleteTablet for tablet 01134b7da20a4915a8a8b9e418646cc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:51 UTC) from {username='slave'} at 127.0.0.1:54426
I20260926 20:50:51.209653 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23217
I20260926 20:50:51.221745 23652 tablet_replica.cc:330] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: stopping tablet replica
I20260926 20:50:51.223786 23652 raft_consensus.cc:2229] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:50:51.225487 23651 tablet_replica.cc:330] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: stopping tablet replica
I20260926 20:50:51.225927 23651 raft_consensus.cc:2229] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:51.226480 23651 raft_consensus.cc:2258] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:51.227391 23650 tablet_replica.cc:330] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: stopping tablet replica
I20260926 20:50:51.227843 23650 raft_consensus.cc:2229] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:51.228371 23650 raft_consensus.cc:2258] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:51.229446 23652 raft_consensus.cc:2258] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:51.230707 23651 ts_tablet_manager.cc:1905] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:51.231812 23652 ts_tablet_manager.cc:1905] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:51.238982 23650 ts_tablet_manager.cc:1905] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:51.239141 23651 ts_tablet_manager.cc:1918] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:51.239298 23652 ts_tablet_manager.cc:1918] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:51.239384 23651 log.cc:1196] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/01134b7da20a4915a8a8b9e418646cc3
I20260926 20:50:51.239986 23651 ts_tablet_manager.cc:1939] T 01134b7da20a4915a8a8b9e418646cc3 P 950bf59cd5ab42c382347cae4e815b87: Deleting consensus metadata
I20260926 20:50:51.240427 23652 log.cc:1196] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/01134b7da20a4915a8a8b9e418646cc3
I20260926 20:50:51.241087 23652 ts_tablet_manager.cc:1939] T 01134b7da20a4915a8a8b9e418646cc3 P 902e4f172eb14721b616e3a9c34b7de3: Deleting consensus metadata
I20260926 20:50:51.242139 23142 catalog_manager.cc:4841] TS 950bf59cd5ab42c382347cae4e815b87 (127.5.78.3:35083): tablet 01134b7da20a4915a8a8b9e418646cc3 (table test-table [id=ea9f53f889ce4d4ba5fd3bb6680dd91c]) successfully deleted
I20260926 20:50:51.242920 23142 catalog_manager.cc:4841] TS 902e4f172eb14721b616e3a9c34b7de3 (127.5.78.2:32805): tablet 01134b7da20a4915a8a8b9e418646cc3 (table test-table [id=ea9f53f889ce4d4ba5fd3bb6680dd91c]) successfully deleted
I20260926 20:50:51.245684 23650 ts_tablet_manager.cc:1918] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:51.245862 23650 log.cc:1196] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/01134b7da20a4915a8a8b9e418646cc3
I20260926 20:50:51.246381 23650 ts_tablet_manager.cc:1939] T 01134b7da20a4915a8a8b9e418646cc3 P 91fed739af014aa1ad9a652b41af8668: Deleting consensus metadata
I20260926 20:50:51.246749 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.211029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.5.78.1:36379)
0926 20:50:51.211237 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.211245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.222947 (+ 11702us) server_negotiation.cc:407] Connection header received
0926 20:50:51.224436 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.224446 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.224604 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.224711 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:51.226189 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.227417 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.229347 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.230037 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.230180 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.244366 (+ 14186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:51.244379 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:51.244399 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:51.245232 (+ 833us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:51.245301 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:51.245331 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:51.245460 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.246101 (+ 641us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.246228 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:50:51.247560 23339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:51.248337 23143 catalog_manager.cc:4841] TS 91fed739af014aa1ad9a652b41af8668 (127.5.78.1:36379): tablet 01134b7da20a4915a8a8b9e418646cc3 (table test-table [id=ea9f53f889ce4d4ba5fd3bb6680dd91c]) successfully deleted
I20260926 20:50:51.355074 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23217
I20260926 20:50:51.376715 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23348
I20260926 20:50:51.384902 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.377627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.5.78.2:32805)
0926 20:50:51.377830 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.377836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.377856 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:50:51.378002 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.378008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.378131 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.378262 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:51.378782 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.379912 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.381288 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.381967 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.382092 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.383237 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:51.383249 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:51.383270 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:51.384113 (+ 843us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:51.384175 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:51.384200 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:51.384313 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.384458 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.384587 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:50:51.385671 23470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:51.499270 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23348
I20260926 20:50:51.518180 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23478
I20260926 20:50:51.525121 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:51.519012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.5.78.3:35083)
0926 20:50:51.519197 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:51.519202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:51.519220 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:50:51.519368 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:51.519372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:51.519452 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:51.519545 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:51.519916 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.520592 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.522029 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:51.522588 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:51.522703 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:51.523788 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:51.523797 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:51.523812 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:51.524374 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:51.524423 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:51.524443 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:51.524545 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:51.524701 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 20:50:51.524784 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:50:51.525805 23600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:51.616196 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23478
I20260926 20:50:51.633178 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23127
I20260926 20:50:51.634295 23188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:51.744385 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23127
2026-09-26T20:50:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3096 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:50:51 dist-test-slave-z7j8 krb5kdc[23660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:50:51 dist-test-slave-z7j8 krb5kdc[23660](info): set up 2 sockets
Sep 26 20:50:51 dist-test-slave-z7j8 krb5kdc[23660](info): commencing operation
krb5kdc: starting...
W20260926 20:50:52.870836 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:50:53 dist-test-slave-z7j8 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:50:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:50:53.121944 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:32913
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:32913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:53.449935 23673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:53.450237 23673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:53.450398 23673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:53.457931 23673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:50:53.458061 23673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:53.458155 23673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:53.458253 23673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:50:53.458339 23673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:50:53.468717 23673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:32913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:32913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:53.470124 23673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:53.471540 23673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:53.483690 23678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:53.483706 23679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:53.483680 23681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:53.484318 23673 server_base.cc:1034] running on GCE node
I20260926 20:50:53.485214 23673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:53.486760 23673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:53.487990 23673 hybrid_clock.cc:648] HybridClock initialized: now 1790455853487936 us; error 33 us; skew 500 ppm
I20260926 20:50:53.488386 23673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:53 dist-test-slave-z7j8 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455853, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:53.491691 23673 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:50:53.493269 23673 webserver.cc:460] Webserver started at http://127.5.78.62:37231/ using document root <none> and password file <none>
I20260926 20:50:53.494002 23673 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:53.494202 23673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:53.494591 23673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:53.497663 23673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "3fc89928a12c4d5ea20d66c9c440729c"
format_stamp: "Formatted at 2026-09-26 20:50:53 on dist-test-slave-z7j8"
server_key: "475f816a4b8e2fd2b0b2db9f851b9aab"
server_key_iv: "60a7653f6875c24f7839e94c21c9ff07"
server_key_version: "encryptionkey@0"
I20260926 20:50:53.498471 23673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "3fc89928a12c4d5ea20d66c9c440729c"
format_stamp: "Formatted at 2026-09-26 20:50:53 on dist-test-slave-z7j8"
server_key: "475f816a4b8e2fd2b0b2db9f851b9aab"
server_key_iv: "60a7653f6875c24f7839e94c21c9ff07"
server_key_version: "encryptionkey@0"
I20260926 20:50:53.504045 23673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 20:50:53.508162 23688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:53.509779 23673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:50:53.510003 23673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "3fc89928a12c4d5ea20d66c9c440729c"
format_stamp: "Formatted at 2026-09-26 20:50:53 on dist-test-slave-z7j8"
server_key: "475f816a4b8e2fd2b0b2db9f851b9aab"
server_key_iv: "60a7653f6875c24f7839e94c21c9ff07"
server_key_version: "encryptionkey@0"
I20260926 20:50:53.510260 23673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:53.524261 23673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:53.528833 23673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:53.529199 23673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:53.550817 23673 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:32913
I20260926 20:50:53.550849 23739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:32913 every 8 connection(s)
I20260926 20:50:53.552508 23673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:50:53.555673 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23673
I20260926 20:50:53.555861 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:50:53.556449 5432 external_mini_cluster.cc:1428] Setting key 6d75ab4061a405f89a98f1b5af31b081
I20260926 20:50:53.557166 23740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:53.570576 23740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c: Bootstrap starting.
Sep 26 20:50:53 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:53.574265 23740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:53.575489 23740 log.cc:824] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:53.578456 23740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c: No bootstrap required, opened a new log
I20260926 20:50:53.580291 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:53.558582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.5.78.62:32913)
0926 20:50:53.559487 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:53.559499 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:53.559546 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:50:53.561212 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:53.561247 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:53.561749 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:53.562257 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:53.563879 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:53.565285 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:53.566839 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:53.567631 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:53.567886 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:53.572294 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:53.572303 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:53.572323 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:53.572369 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:53.575950 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:53.576805 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:53.576812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:53.576820 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:53.576994 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:53.577417 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:53.577421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:53.577423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:53.577966 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:53.578290 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:53.578814 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 20:50:53.579261 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":397,"threads_started":1}
I20260926 20:50:53.586987 23740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc89928a12c4d5ea20d66c9c440729c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32913 } }
I20260926 20:50:53.587450 23740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:53.587561 23740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc89928a12c4d5ea20d66c9c440729c, State: Initialized, Role: FOLLOWER
I20260926 20:50:53.588128 23740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc89928a12c4d5ea20d66c9c440729c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32913 } }
I20260926 20:50:53.588382 23740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:50:53.588522 23740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:50:53.588704 23740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:53.591033 23740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc89928a12c4d5ea20d66c9c440729c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32913 } }
I20260926 20:50:53.591568 23740 leader_election.cc:302] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fc89928a12c4d5ea20d66c9c440729c; no voters:
I20260926 20:50:53.592340 23740 leader_election.cc:288] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:50:53.592479 23745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:53.593237 23745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [term 1 LEADER]: Becoming Leader. State: Replica: 3fc89928a12c4d5ea20d66c9c440729c, State: Running, Role: LEADER
I20260926 20:50:53.593858 23745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc89928a12c4d5ea20d66c9c440729c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32913 } }
I20260926 20:50:53.594599 23740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:50:53.596820 23746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fc89928a12c4d5ea20d66c9c440729c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc89928a12c4d5ea20d66c9c440729c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32913 } } }
I20260926 20:50:53.597072 23747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc89928a12c4d5ea20d66c9c440729c. Latest consensus state: current_term: 1 leader_uuid: "3fc89928a12c4d5ea20d66c9c440729c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc89928a12c4d5ea20d66c9c440729c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 32913 } } }
I20260926 20:50:53.597309 23746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:53.597362 23747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:53.599347 23754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:50:53.603909 23754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:50:53.614030 23754 catalog_manager.cc:1347] Generated new cluster ID: a333652808de41419c19098a8d3ecbea
I20260926 20:50:53.614176 23754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:50:53.636456 23754 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:50:53.637894 23754 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:50:53.648198 23754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c: Generated new TSK 0
I20260926 20:50:53.649160 23754 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:50:53.714836 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32913
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:54.061600 23766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:54.061918 23766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:54.062088 23766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:54.069658 23766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:54.069789 23766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:54.069980 23766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:50:54.079927 23766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:54.081286 23766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:54.082667 23766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:54.096390 23772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:54.096906 23771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:54.096421 23774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:54.097711 23766 server_base.cc:1034] running on GCE node
I20260926 20:50:54.098368 23766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:54.099509 23766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:54.100736 23766 hybrid_clock.cc:648] HybridClock initialized: now 1790455854100675 us; error 53 us; skew 500 ppm
I20260926 20:50:54.101049 23766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:54 dist-test-slave-z7j8 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455854, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:54.104337 23766 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:50:54.105800 23766 webserver.cc:460] Webserver started at http://127.5.78.1:39027/ using document root <none> and password file <none>
I20260926 20:50:54.106563 23766 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:54.106714 23766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:54.107059 23766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:54.110097 23766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "f4e466ec14904170a6f978f127cde12d"
format_stamp: "Formatted at 2026-09-26 20:50:54 on dist-test-slave-z7j8"
server_key: "96064fd7e27c596c9520aa486c8cb074"
server_key_iv: "dbdd8072a9b76920c47149b2f85ac444"
server_key_version: "encryptionkey@0"
I20260926 20:50:54.110956 23766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "f4e466ec14904170a6f978f127cde12d"
format_stamp: "Formatted at 2026-09-26 20:50:54 on dist-test-slave-z7j8"
server_key: "96064fd7e27c596c9520aa486c8cb074"
server_key_iv: "dbdd8072a9b76920c47149b2f85ac444"
server_key_version: "encryptionkey@0"
I20260926 20:50:54.116814 23766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:50:54.120559 23781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:54.121955 23766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:50:54.122130 23766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "f4e466ec14904170a6f978f127cde12d"
format_stamp: "Formatted at 2026-09-26 20:50:54 on dist-test-slave-z7j8"
server_key: "96064fd7e27c596c9520aa486c8cb074"
server_key_iv: "dbdd8072a9b76920c47149b2f85ac444"
server_key_version: "encryptionkey@0"
I20260926 20:50:54.122340 23766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:54.138360 23766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:54.142731 23766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:54.143023 23766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:54.144016 23766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:54.145393 23766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:54.145503 23766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:54.145666 23766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:54.145740 23766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:54.181509 23766 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:33709
I20260926 20:50:54.181646 23893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:33709 every 8 connection(s)
I20260926 20:50:54.182866 23766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:50:54.183872 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23766
I20260926 20:50:54.184055 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:50:54.184579 5432 external_mini_cluster.cc:1428] Setting key bc2c65fdc8567346bf0a806246a69a5e
Sep 26 20:50:54 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455854, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:54.204860 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:54.185656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:39219 (local address 127.5.78.62:32913)
0926 20:50:54.186109 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:54.186115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:54.187247 (+ 1132us) server_negotiation.cc:407] Connection header received
0926 20:50:54.189079 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:54.189086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:54.189178 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:54.189298 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:54.190908 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.192106 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.193633 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.194233 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.194371 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:54.198816 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:54.198822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:54.198825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:54.198859 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:54.201657 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:54.202445 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:54.202453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:54.202460 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:54.202579 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:54.203053 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:54.203058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:54.203060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:54.203351 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:54.203516 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:54.204252 (+ 736us) server_negotiation.cc:299] Negotiation successful
0926 20:50:54.204482 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260926 20:50:54.205794 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:54.186183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32913 (local address 127.5.78.1:39219)
0926 20:50:54.187068 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:54.187131 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:54.188817 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:54.189556 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:54.189564 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:54.190074 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:54.190733 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:54.190754 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.192255 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.192259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:54.193484 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:54.193495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.194515 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.194525 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:54.194903 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:54.195742 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:54.195779 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:54.198567 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:54.201815 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:54.201824 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:54.201842 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:54.202282 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:54.202717 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:54.202722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:54.202726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:54.202922 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:54.203516 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:54.203525 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:54.203869 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 20:50:54.204150 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 20:50:54.204690 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":384,"threads_started":1}
I20260926 20:50:54.207368 23894 heartbeater.cc:347] Connected to a master server at 127.5.78.62:32913
I20260926 20:50:54.207737 23894 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:54.208688 23894 heartbeater.cc:511] Master 127.5.78.62:32913 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
I20260926 20:50:54.211053 23705 ts_manager.cc:194] Registered new tserver with Master: f4e466ec14904170a6f978f127cde12d (127.5.78.1:33709)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
I20260926 20:50:54.212638 23705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:39219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:50:54.239949 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32913
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:54.566138 23900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:54.566521 23900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:54.566636 23900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:54.574079 23900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:54.574214 23900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:54.574335 23900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:50:54.584542 23900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:54.585798 23900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:54.587153 23900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:54.600423 23905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:54.600545 23908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:54.600571 23906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:54.601663 23900 server_base.cc:1034] running on GCE node
I20260926 20:50:54.602356 23900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:54.603452 23900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:54.604720 23900 hybrid_clock.cc:648] HybridClock initialized: now 1790455854604667 us; error 38 us; skew 500 ppm
I20260926 20:50:54.605111 23900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:54 dist-test-slave-z7j8 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455854, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:54.608560 23900 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:50:54.610219 23900 webserver.cc:460] Webserver started at http://127.5.78.2:40299/ using document root <none> and password file <none>
I20260926 20:50:54.610989 23900 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:54.611095 23900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:54.611357 23900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:54.614751 23900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "32aba904b6a248718eae6e37ff5cc26f"
format_stamp: "Formatted at 2026-09-26 20:50:54 on dist-test-slave-z7j8"
server_key: "75f970530a9480c9ca627c5e890e8494"
server_key_iv: "69cf9991a87b007b98dcd4af20dcd001"
server_key_version: "encryptionkey@0"
I20260926 20:50:54.615628 23900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "32aba904b6a248718eae6e37ff5cc26f"
format_stamp: "Formatted at 2026-09-26 20:50:54 on dist-test-slave-z7j8"
server_key: "75f970530a9480c9ca627c5e890e8494"
server_key_iv: "69cf9991a87b007b98dcd4af20dcd001"
server_key_version: "encryptionkey@0"
I20260926 20:50:54.620893 23900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:50:54.624523 23915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:54.626284 23900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:50:54.626480 23900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "32aba904b6a248718eae6e37ff5cc26f"
format_stamp: "Formatted at 2026-09-26 20:50:54 on dist-test-slave-z7j8"
server_key: "75f970530a9480c9ca627c5e890e8494"
server_key_iv: "69cf9991a87b007b98dcd4af20dcd001"
server_key_version: "encryptionkey@0"
I20260926 20:50:54.626729 23900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:54.650812 23900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:54.655512 23900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:54.655839 23900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:54.656864 23900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:54.658218 23900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:54.658293 23900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:54.658440 23900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:54.658491 23900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:50:54.691751 23900 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:42971
I20260926 20:50:54.691833 24027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:42971 every 8 connection(s)
I20260926 20:50:54.693063 23900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:50:54.694550 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 23900
I20260926 20:50:54.694772 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:50:54.695325 5432 external_mini_cluster.cc:1428] Setting key 5fd35a7920beaae3e0485674a324aebe
Sep 26 20:50:54 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455854, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:54.718583 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:54.696016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:48459 (local address 127.5.78.62:32913)
0926 20:50:54.696210 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:54.696221 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:54.697625 (+ 1404us) server_negotiation.cc:407] Connection header received
0926 20:50:54.700612 (+ 2987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:54.700617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:54.700687 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:54.700780 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:54.702900 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.703619 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.706141 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.706904 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.707029 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:54.711215 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:54.711225 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:54.711233 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:54.711287 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:54.715242 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:54.716086 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:54.716091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:54.716093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:54.716229 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:54.716731 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:54.716735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:54.716737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:54.717015 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:54.717178 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:54.717935 (+ 757us) server_negotiation.cc:299] Negotiation successful
0926 20:50:54.718172 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:50:54.719830 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:54.696637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32913 (local address 127.5.78.2:48459)
0926 20:50:54.697454 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:54.697502 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:54.700303 (+ 2801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:54.701058 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:54.701073 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:54.701869 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:54.702716 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:54.702745 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.703782 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.703790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:54.705957 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:54.705974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:54.707154 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:54.707165 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:54.707402 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:54.708184 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:54.708210 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:54.710889 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:54.715415 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:54.715422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:54.715437 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:54.715918 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:54.716355 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:54.716359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:54.716370 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:54.716571 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:54.717179 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:54.717188 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:54.717558 (+ 370us) client_negotiation.cc:769] Sending connection context
0926 20:50:54.717841 (+ 283us) client_negotiation.cc:240] Negotiation successful
0926 20:50:54.718490 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":304,"threads_started":1}
I20260926 20:50:54.721396 24028 heartbeater.cc:347] Connected to a master server at 127.5.78.62:32913
I20260926 20:50:54.721726 24028 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:54.722496 24028 heartbeater.cc:511] Master 127.5.78.62:32913 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
I20260926 20:50:54.724006 23705 ts_manager.cc:194] Registered new tserver with Master: 32aba904b6a248718eae6e37ff5cc26f (127.5.78.2:42971)
I20260926 20:50:54.724809 23705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:48459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:50:54.755419 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32913
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:50:55.092464 24033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:55.092831 24033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:55.092999 24033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:55.100888 24033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:55.101051 24033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:55.101225 24033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:50:55.111848 24033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:32913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:55.113253 24033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:55.114853 24033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:55.128155 24038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:55.128226 24039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:55.128366 24041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:55.129020 24033 server_base.cc:1034] running on GCE node
I20260926 20:50:55.129671 24033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:55.130807 24033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:55.132043 24033 hybrid_clock.cc:648] HybridClock initialized: now 1790455855131972 us; error 56 us; skew 500 ppm
I20260926 20:50:55.132421 24033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:50:55 dist-test-slave-z7j8 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455855, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:50:55.135802 24033 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:50:55.137357 24033 webserver.cc:460] Webserver started at http://127.5.78.3:35641/ using document root <none> and password file <none>
I20260926 20:50:55.138027 24033 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:55.138125 24033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:55.138391 24033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:55.141487 24033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "0401b7e10c4045f2acc673f5187dad1d"
format_stamp: "Formatted at 2026-09-26 20:50:55 on dist-test-slave-z7j8"
server_key: "333915e6371f19f63d1e885000eade44"
server_key_iv: "25cf66319c9a2c56ee1de4146ba81777"
server_key_version: "encryptionkey@0"
I20260926 20:50:55.142633 24033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "0401b7e10c4045f2acc673f5187dad1d"
format_stamp: "Formatted at 2026-09-26 20:50:55 on dist-test-slave-z7j8"
server_key: "333915e6371f19f63d1e885000eade44"
server_key_iv: "25cf66319c9a2c56ee1de4146ba81777"
server_key_version: "encryptionkey@0"
I20260926 20:50:55.148540 24033 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260926 20:50:55.152045 24048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:55.153419 24033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:50:55.153553 24033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "0401b7e10c4045f2acc673f5187dad1d"
format_stamp: "Formatted at 2026-09-26 20:50:55 on dist-test-slave-z7j8"
server_key: "333915e6371f19f63d1e885000eade44"
server_key_iv: "25cf66319c9a2c56ee1de4146ba81777"
server_key_version: "encryptionkey@0"
I20260926 20:50:55.153729 24033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:55.167713 24033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:55.172130 24033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:55.172433 24033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:55.173372 24033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:55.174944 24033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:55.175083 24033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 20:50:55.175225 24033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:55.175312 24033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:55.210707 24033 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:38503
I20260926 20:50:55.210816 24160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:38503 every 8 connection(s)
I20260926 20:50:55.211966 24033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:50:55.216274 23894 heartbeater.cc:503] Master 127.5.78.62:32913 was elected leader, sending a full tablet report...
I20260926 20:50:55.220943 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24033
I20260926 20:50:55.221149 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:50:55.221585 5432 external_mini_cluster.cc:1428] Setting key 19133fcc1d3533dc1734a27a2ac0f46e
Sep 26 20:50:55 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455855, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:50:55.233171 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.214712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:52695 (local address 127.5.78.62:32913)
0926 20:50:55.214897 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.214901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.216161 (+ 1260us) server_negotiation.cc:407] Connection header received
0926 20:50:55.218620 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.218625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.218707 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.218820 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:55.220291 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.221112 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.223115 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.223664 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.223787 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.227485 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:55.227489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:55.227492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:55.227532 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:55.230118 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.230978 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.230983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.230985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.231110 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.231529 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.231533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.231536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.231830 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:55.231948 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.232707 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.232883 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:50:55.234141 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.215229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:32913 (local address 127.5.78.3:52695)
0926 20:50:55.215987 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:55.216026 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:55.218303 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:55.219052 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:55.219060 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:55.219566 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:55.220121 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.220138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.221261 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.221268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.222962 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.222974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.223882 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.223887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.224153 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.224892 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:50:55.224914 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:55.227298 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:55.230275 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:55.230282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:55.230304 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:55.230834 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:55.231217 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:50:55.231221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:50:55.231223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:50:55.231425 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:50:55.231972 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:55.231978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:55.232316 (+ 338us) client_negotiation.cc:769] Sending connection context
0926 20:50:55.232550 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 20:50:55.232962 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":303,"threads_started":1}
I20260926 20:50:55.235812 24161 heartbeater.cc:347] Connected to a master server at 127.5.78.62:32913
I20260926 20:50:55.236155 24161 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:55.236994 24161 heartbeater.cc:511] Master 127.5.78.62:32913 requested a full tablet report, sending...
I20260926 20:50:55.238476 23705 ts_manager.cc:194] Registered new tserver with Master: 0401b7e10c4045f2acc673f5187dad1d (127.5.78.3:38503)
I20260926 20:50:55.239270 23705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:52695
I20260926 20:50:55.249339 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:50:55.267611 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.253724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.5.78.62:32913)
0926 20:50:55.253970 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.253979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.254389 (+ 410us) server_negotiation.cc:407] Connection header received
0926 20:50:55.254642 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.254653 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.254718 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.254823 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:55.256270 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.257060 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.258626 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.259030 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.259200 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.261893 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:55.261899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:55.261911 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:55.261952 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:55.264596 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.265418 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.265423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.265427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.265566 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.266035 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.266040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.266043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.266368 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:55.266708 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.267076 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.267324 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:50:55.272059 23705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:50:55.279189 23705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:50:55.304371 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.294298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.5.78.3:38503)
0926 20:50:55.295348 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.295360 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.295401 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:50:55.295523 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.295535 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.295826 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.296113 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:55.296895 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.298615 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.301500 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.302951 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.303506 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.303758 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.303845 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.304012 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":655,"threads_started":1}
I20260926 20:50:55.304405 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.293973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:38503 (local address 127.0.0.1:59552)
0926 20:50:55.294554 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:55.294571 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:55.294678 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:55.296108 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:55.296115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:55.296146 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:55.296637 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.296660 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.298771 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.298779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.301339 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.301356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.303201 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.303207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.303494 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.303559 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 20:50:55.303702 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 20:50:55.303942 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":75,"thread_start_us":293,"threads_started":1}
I20260926 20:50:55.304605 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.292485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46778 (local address 127.5.78.1:33709)
0926 20:50:55.294001 (+ 1516us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.294014 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.294062 (+ 48us) server_negotiation.cc:407] Connection header received
0926 20:50:55.294228 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.294237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.295488 (+ 1251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.296638 (+ 1150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:55.296917 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.298672 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.301358 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.302951 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.303102 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.303382 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.304080 (+ 698us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.304273 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1342,"thread_start_us":1180,"threads_started":1}
I20260926 20:50:55.305299 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.292278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:33709 (local address 127.0.0.1:46778)
0926 20:50:55.293105 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:55.293162 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:55.293352 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:55.295755 (+ 2403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:55.295765 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:55.295803 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:55.296337 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.296348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.298835 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.298844 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.301152 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.301166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.303146 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.303152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.303489 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.303557 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 20:50:55.303701 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 20:50:55.303945 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":94,"thread_start_us":740,"threads_started":1}
I20260926 20:50:55.305716 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.294616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:42971 (local address 127.0.0.1:37720)
0926 20:50:55.298070 (+ 3454us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:55.298084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:55.298226 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:55.298900 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:55.298906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:55.298938 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:55.299513 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.299531 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.301459 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.301466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.302834 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.302845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.304911 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.304916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.305067 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.305097 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 20:50:55.305186 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:50:55.305345 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3310,"mutex_wait_us":1,"thread_start_us":3191,"threads_started":1}
I20260926 20:50:55.306663 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.294968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.5.78.2:42971)
0926 20:50:55.295941 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.295951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.298193 (+ 2242us) server_negotiation.cc:407] Connection header received
0926 20:50:55.298444 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.298454 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.298663 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.298883 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:55.299691 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.301286 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.302988 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.304516 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.304654 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.306092 (+ 1438us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.306164 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.306317 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":277,"threads_started":1}
I20260926 20:50:55.309531 24096 tablet_service.cc:1467] Processing CreateTablet for tablet daabf7f795834d40a7fe9db29b4e25a3 (DEFAULT_TABLE table=test-table [id=86b8e55cf77d4a4d9f633f4ea7d04b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:55.309619 23829 tablet_service.cc:1467] Processing CreateTablet for tablet daabf7f795834d40a7fe9db29b4e25a3 (DEFAULT_TABLE table=test-table [id=86b8e55cf77d4a4d9f633f4ea7d04b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:55.309857 23963 tablet_service.cc:1467] Processing CreateTablet for tablet daabf7f795834d40a7fe9db29b4e25a3 (DEFAULT_TABLE table=test-table [id=86b8e55cf77d4a4d9f633f4ea7d04b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:55.311108 23829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daabf7f795834d40a7fe9db29b4e25a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:55.311106 23963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daabf7f795834d40a7fe9db29b4e25a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:55.311108 24096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daabf7f795834d40a7fe9db29b4e25a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:55.323499 24179 tablet_bootstrap.cc:492] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Bootstrap starting.
I20260926 20:50:55.325736 24181 tablet_bootstrap.cc:492] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Bootstrap starting.
I20260926 20:50:55.329070 24179 tablet_bootstrap.cc:654] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:55.329084 24181 tablet_bootstrap.cc:654] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:55.330487 24179 log.cc:824] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:55.330488 24181 log.cc:824] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:55.332809 24180 tablet_bootstrap.cc:492] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Bootstrap starting.
I20260926 20:50:55.334779 24179 tablet_bootstrap.cc:492] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: No bootstrap required, opened a new log
I20260926 20:50:55.335175 24179 ts_tablet_manager.cc:1397] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260926 20:50:55.336431 24181 tablet_bootstrap.cc:492] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: No bootstrap required, opened a new log
I20260926 20:50:55.336804 24181 ts_tablet_manager.cc:1397] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 20:50:55.337877 24180 tablet_bootstrap.cc:654] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:55.339265 24180 log.cc:824] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:55.345073 24180 tablet_bootstrap.cc:492] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: No bootstrap required, opened a new log
I20260926 20:50:55.345410 24180 ts_tablet_manager.cc:1397] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.005s
I20260926 20:50:55.346158 24179 raft_consensus.cc:348] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.346745 24179 raft_consensus.cc:374] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:55.346884 24179 raft_consensus.cc:729] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0401b7e10c4045f2acc673f5187dad1d, State: Initialized, Role: FOLLOWER
I20260926 20:50:55.347553 24179 consensus_queue.cc:260] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.347961 24181 raft_consensus.cc:348] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.348419 24181 raft_consensus.cc:374] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:55.348520 24181 raft_consensus.cc:729] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e466ec14904170a6f978f127cde12d, State: Initialized, Role: FOLLOWER
I20260926 20:50:55.348937 24161 heartbeater.cc:503] Master 127.5.78.62:32913 was elected leader, sending a full tablet report...
I20260926 20:50:55.349241 24181 consensus_queue.cc:260] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.349553 24179 ts_tablet_manager.cc:1428] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 20:50:55.351248 24181 ts_tablet_manager.cc:1428] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260926 20:50:55.357218 24180 raft_consensus.cc:348] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.357707 24180 raft_consensus.cc:374] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:55.357820 24180 raft_consensus.cc:729] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32aba904b6a248718eae6e37ff5cc26f, State: Initialized, Role: FOLLOWER
I20260926 20:50:55.358392 24180 consensus_queue.cc:260] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.359509 24028 heartbeater.cc:503] Master 127.5.78.62:32913 was elected leader, sending a full tablet report...
I20260926 20:50:55.360224 24180 ts_tablet_manager.cc:1428] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260926 20:50:55.435313 23895 tablet.cc:2369] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:55.445451 24029 tablet.cc:2369] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:55.463619 24162 tablet.cc:2369] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:55.480252 24185 raft_consensus.cc:482] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:50:55.480554 24185 raft_consensus.cc:504] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.482299 24185 leader_election.cc:288] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e466ec14904170a6f978f127cde12d (127.5.78.1:33709), 32aba904b6a248718eae6e37ff5cc26f (127.5.78.2:42971)
I20260926 20:50:55.489257 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.482779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:44417 (local address 127.5.78.1:33709)
0926 20:50:55.482971 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.482975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.483017 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:50:55.483554 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.483560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.483642 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.483752 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:55.484291 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.485271 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.487309 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.488357 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.488492 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.488596 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.488878 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.489012 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:50:55.489295 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.482663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:33709 (local address 127.5.78.3:44417)
0926 20:50:55.482899 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:55.482921 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:55.483058 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:55.483784 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:55.483787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:55.483809 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:55.484099 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.484111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.485418 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.485424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.487129 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.487146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.488515 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.488521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.488703 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.488730 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 20:50:55.488837 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:50:55.488976 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"spinlock_wait_cycles":37248}
I20260926 20:50:55.489881 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.483376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:59477 (local address 127.5.78.2:42971)
0926 20:50:55.483608 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.483617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.483887 (+ 270us) server_negotiation.cc:407] Connection header received
0926 20:50:55.484071 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.484076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.484153 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.484250 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:55.484904 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.485812 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.487794 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.488890 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.489055 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.489161 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.489500 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.489629 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:50:55.490617 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.483108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:42971 (local address 127.5.78.3:59477)
0926 20:50:55.483783 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:55.483801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:55.483941 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:55.484279 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:55.484283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:55.484307 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:55.484702 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.484714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.485990 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.485994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.487604 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:55.487616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.489107 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.489113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:55.489286 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.489309 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:50:55.490091 (+ 782us) client_negotiation.cc:240] Negotiation successful
0926 20:50:55.490240 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":317,"threads_started":1}
I20260926 20:50:55.490459 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daabf7f795834d40a7fe9db29b4e25a3" candidate_uuid: "0401b7e10c4045f2acc673f5187dad1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e466ec14904170a6f978f127cde12d" is_pre_election: true
I20260926 20:50:55.491110 23849 raft_consensus.cc:2454] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0401b7e10c4045f2acc673f5187dad1d in term 0.
I20260926 20:50:55.491766 23983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daabf7f795834d40a7fe9db29b4e25a3" candidate_uuid: "0401b7e10c4045f2acc673f5187dad1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32aba904b6a248718eae6e37ff5cc26f" is_pre_election: true
I20260926 20:50:55.491894 24051 leader_election.cc:302] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0401b7e10c4045f2acc673f5187dad1d, f4e466ec14904170a6f978f127cde12d; no voters:
I20260926 20:50:55.492355 24185 raft_consensus.cc:2781] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:50:55.492359 23983 raft_consensus.cc:2454] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0401b7e10c4045f2acc673f5187dad1d in term 0.
I20260926 20:50:55.492548 24185 raft_consensus.cc:482] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:50:55.492691 24185 raft_consensus.cc:3037] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:55.495476 24185 raft_consensus.cc:504] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.496174 24185 leader_election.cc:288] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [CANDIDATE]: Term 1 election: Requested vote from peers f4e466ec14904170a6f978f127cde12d (127.5.78.1:33709), 32aba904b6a248718eae6e37ff5cc26f (127.5.78.2:42971)
I20260926 20:50:55.496765 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daabf7f795834d40a7fe9db29b4e25a3" candidate_uuid: "0401b7e10c4045f2acc673f5187dad1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e466ec14904170a6f978f127cde12d"
I20260926 20:50:55.496830 23983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daabf7f795834d40a7fe9db29b4e25a3" candidate_uuid: "0401b7e10c4045f2acc673f5187dad1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32aba904b6a248718eae6e37ff5cc26f"
I20260926 20:50:55.497006 23849 raft_consensus.cc:3037] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:55.497044 23983 raft_consensus.cc:3037] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:55.499493 23983 raft_consensus.cc:2454] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0401b7e10c4045f2acc673f5187dad1d in term 1.
I20260926 20:50:55.499748 23849 raft_consensus.cc:2454] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0401b7e10c4045f2acc673f5187dad1d in term 1.
I20260926 20:50:55.500029 24051 leader_election.cc:302] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0401b7e10c4045f2acc673f5187dad1d, 32aba904b6a248718eae6e37ff5cc26f; no voters:
I20260926 20:50:55.500449 24185 raft_consensus.cc:2781] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:55.500996 24185 raft_consensus.cc:686] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 1 LEADER]: Becoming Leader. State: Replica: 0401b7e10c4045f2acc673f5187dad1d, State: Running, Role: LEADER
I20260926 20:50:55.501497 24185 consensus_queue.cc:237] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } }
I20260926 20:50:55.506152 23705 catalog_manager.cc:5495] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0401b7e10c4045f2acc673f5187dad1d (127.5.78.3). New cstate: current_term: 1 leader_uuid: "0401b7e10c4045f2acc673f5187dad1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0401b7e10c4045f2acc673f5187dad1d" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 38503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 } health_report { overall_health: UNKNOWN } } }
I20260926 20:50:55.547545 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.541858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.5.78.3:38503)
0926 20:50:55.542056 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.542061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.542086 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:50:55.542238 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.542242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.542322 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.542543 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:50:55.542935 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.543683 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.545175 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.545597 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.545738 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.546358 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:50:55.546988 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:50:55.547119 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.547287 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.547336 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:50:55.557456 23983 raft_consensus.cc:1261] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 1 FOLLOWER]: Refusing update from remote peer 0401b7e10c4045f2acc673f5187dad1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:55.557468 23849 raft_consensus.cc:1261] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 1 FOLLOWER]: Refusing update from remote peer 0401b7e10c4045f2acc673f5187dad1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:55.558632 24185 consensus_queue.cc:1035] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32aba904b6a248718eae6e37ff5cc26f" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 42971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:55.558993 24189 consensus_queue.cc:1035] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e466ec14904170a6f978f127cde12d" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:55.588465 23704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:55.588768 23704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:55.592545 23704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fc89928a12c4d5ea20d66c9c440729c: Sending DeleteTablet for 3 replicas of tablet daabf7f795834d40a7fe9db29b4e25a3
I20260926 20:50:55.593930 24096 tablet_service.cc:1514] Processing DeleteTablet for tablet daabf7f795834d40a7fe9db29b4e25a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:55 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:59552
I20260926 20:50:55.594069 23963 tablet_service.cc:1514] Processing DeleteTablet for tablet daabf7f795834d40a7fe9db29b4e25a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:55 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:37720
I20260926 20:50:55.594137 23829 tablet_service.cc:1514] Processing DeleteTablet for tablet daabf7f795834d40a7fe9db29b4e25a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:55 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:46778
I20260926 20:50:55.598935 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23766
I20260926 20:50:55.607025 24204 tablet_replica.cc:330] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: stopping tablet replica
I20260926 20:50:55.607734 24204 raft_consensus.cc:2229] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:55.608457 24204 raft_consensus.cc:2258] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:55.609163 24203 tablet_replica.cc:330] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: stopping tablet replica
I20260926 20:50:55.611843 24204 ts_tablet_manager.cc:1905] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:55.612418 24203 raft_consensus.cc:2229] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:50:55.613219 24203 raft_consensus.cc:2258] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:55.615981 24202 tablet_replica.cc:330] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: stopping tablet replica
I20260926 20:50:55.616374 24202 raft_consensus.cc:2229] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:55.616844 24202 raft_consensus.cc:2258] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 20:50:55 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:50:55.620483 24202 ts_tablet_manager.cc:1905] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:55.622644 24204 ts_tablet_manager.cc:1918] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:55.622859 24204 log.cc:1196] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/daabf7f795834d40a7fe9db29b4e25a3
I20260926 20:50:55.623579 24204 ts_tablet_manager.cc:1939] T daabf7f795834d40a7fe9db29b4e25a3 P f4e466ec14904170a6f978f127cde12d: Deleting consensus metadata
I20260926 20:50:55.625830 23691 catalog_manager.cc:4841] TS f4e466ec14904170a6f978f127cde12d (127.5.78.1:33709): tablet daabf7f795834d40a7fe9db29b4e25a3 (table test-table [id=86b8e55cf77d4a4d9f633f4ea7d04b13]) successfully deleted
I20260926 20:50:55.626595 24203 ts_tablet_manager.cc:1905] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:55.629191 24202 ts_tablet_manager.cc:1918] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:55.629434 24202 log.cc:1196] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/daabf7f795834d40a7fe9db29b4e25a3
I20260926 20:50:55.629753 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.600789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46784 (local address 127.5.78.1:33709)
0926 20:50:55.601016 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.601023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.603263 (+ 2240us) server_negotiation.cc:407] Connection header received
0926 20:50:55.604258 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.604267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.604385 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.604481 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:55.609022 (+ 4541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.610657 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.613344 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.614275 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.614394 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.621552 (+ 7158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:55.621568 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:55.621586 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:55.621633 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:55.625767 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.627085 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.627095 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.627099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.627275 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.627754 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.627758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.627762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.628213 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:55.628480 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.629083 (+ 603us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.629331 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:50:55.630332 24202 ts_tablet_manager.cc:1939] T daabf7f795834d40a7fe9db29b4e25a3 P 32aba904b6a248718eae6e37ff5cc26f: Deleting consensus metadata
I20260926 20:50:55.631321 23889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:55.632365 23691 catalog_manager.cc:4841] TS 32aba904b6a248718eae6e37ff5cc26f (127.5.78.2:42971): tablet daabf7f795834d40a7fe9db29b4e25a3 (table test-table [id=86b8e55cf77d4a4d9f633f4ea7d04b13]) successfully deleted
I20260926 20:50:55.638304 24203 ts_tablet_manager.cc:1918] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:55.638525 24203 log.cc:1196] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/daabf7f795834d40a7fe9db29b4e25a3
I20260926 20:50:55.639094 24203 ts_tablet_manager.cc:1939] T daabf7f795834d40a7fe9db29b4e25a3 P 0401b7e10c4045f2acc673f5187dad1d: Deleting consensus metadata
I20260926 20:50:55.641100 23691 catalog_manager.cc:4841] TS 0401b7e10c4045f2acc673f5187dad1d (127.5.78.3:38503): tablet daabf7f795834d40a7fe9db29b4e25a3 (table test-table [id=86b8e55cf77d4a4d9f633f4ea7d04b13]) successfully deleted
I20260926 20:50:55.732604 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23766
I20260926 20:50:55.752996 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23900
Sep 26 20:50:55 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:50:55.773933 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.753911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37730 (local address 127.5.78.2:42971)
0926 20:50:55.754123 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.754132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.754157 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:50:55.754283 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.754289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.754424 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.754545 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:55.756693 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.757961 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.759937 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.760340 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.760448 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.766007 (+ 5559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:55.766030 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:55.766051 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:55.766114 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:55.770246 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.771287 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.771300 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.771304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.771475 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.771929 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.771935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.771939 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.772493 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:55.772821 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.773180 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.773484 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:50:55.774780 24023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:55.865466 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23900
I20260926 20:50:55.883587 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24033
Sep 26 20:50:55 dist-test-slave-z7j8 krb5kdc[23660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:50:55.899950 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:55.884416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.5.78.3:38503)
0926 20:50:55.884602 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:55.884608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:55.884659 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:50:55.884908 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:55.884912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:55.884986 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:55.885095 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:55.886437 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.887298 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.888638 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:55.889088 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:55.889237 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:55.893698 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:55.893714 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:55.893729 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:50:55.893766 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:55.896832 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.897564 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.897572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.897574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.897751 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:50:55.898144 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:50:55.898149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:50:55.898162 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:50:55.898593 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:55.898948 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:55.899305 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 20:50:55.899537 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:50:55.900933 24156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:56.023216 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24033
I20260926 20:50:56.042939 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 23673
I20260926 20:50:56.044144 23735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:56.155334 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 23673
2026-09-26T20:50:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4412 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T20:50:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:50:56Z Disabled control of system clock
I20260926 20:50:56.235074 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38967
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:50:56.586140 24213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:56.586491 24213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:56.586642 24213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:56.594152 24213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:50:56.594271 24213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:56.594347 24213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:56.594462 24213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:50:56.594579 24213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:50:56.604070 24213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:38967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:56.605415 24213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:56.607368 24213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:56.619824 24219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:56.619876 24221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:56.620007 24218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:56.620929 24213 server_base.cc:1034] running on GCE node
I20260926 20:50:56.621848 24213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:56.623513 24213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:56.624776 24213 hybrid_clock.cc:648] HybridClock initialized: now 1790455856624703 us; error 78 us; skew 500 ppm
I20260926 20:50:56.625169 24213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:56.632277 24213 webserver.cc:460] Webserver started at http://127.5.78.62:40859/ using document root <none> and password file <none>
I20260926 20:50:56.633121 24213 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:56.633281 24213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:56.633639 24213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:56.636943 24213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "e005d8e1347a41578955f4106acf09e7"
format_stamp: "Formatted at 2026-09-26 20:50:56 on dist-test-slave-z7j8"
server_key: "a87400016cee3962873aa143be01ea80"
server_key_iv: "fcece30a43b371d70e22e1ffa3ba0581"
server_key_version: "encryptionkey@0"
I20260926 20:50:56.637811 24213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "e005d8e1347a41578955f4106acf09e7"
format_stamp: "Formatted at 2026-09-26 20:50:56 on dist-test-slave-z7j8"
server_key: "a87400016cee3962873aa143be01ea80"
server_key_iv: "fcece30a43b371d70e22e1ffa3ba0581"
server_key_version: "encryptionkey@0"
I20260926 20:50:56.643729 24213 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:50:56.647954 24227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:56.649608 24213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:50:56.649852 24213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "e005d8e1347a41578955f4106acf09e7"
format_stamp: "Formatted at 2026-09-26 20:50:56 on dist-test-slave-z7j8"
server_key: "a87400016cee3962873aa143be01ea80"
server_key_iv: "fcece30a43b371d70e22e1ffa3ba0581"
server_key_version: "encryptionkey@0"
I20260926 20:50:56.650127 24213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:56.664737 24213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:56.665681 24213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:56.666002 24213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:56.685113 24213 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:38967
I20260926 20:50:56.685145 24278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:38967 every 8 connection(s)
I20260926 20:50:56.686513 24213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:50:56.690227 24279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:56.691448 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24213
I20260926 20:50:56.691634 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:50:56.692070 5432 external_mini_cluster.cc:1428] Setting key 825e2a2b46c41348ad108b69942bc0aa
I20260926 20:50:56.704497 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7: Bootstrap starting.
I20260926 20:50:56.706262 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:56.694069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.5.78.62:38967)
0926 20:50:56.695150 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:56.695161 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:56.695206 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:50:56.696736 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:56.696756 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:56.697231 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:56.697575 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:56.698193 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:56.699532 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:56.701665 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:56.702452 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:56.702610 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:56.703871 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:56.703880 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:56.703895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:56.704468 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:56.704557 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:56.704577 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:56.704701 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:56.704954 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 20:50:56.705275 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":404,"threads_started":1}
I20260926 20:50:56.708058 24279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:56.709436 24279 log.cc:824] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:56.712684 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7: No bootstrap required, opened a new log
I20260926 20:50:56.720799 24279 raft_consensus.cc:348] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e005d8e1347a41578955f4106acf09e7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38967 } }
I20260926 20:50:56.721210 24279 raft_consensus.cc:374] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:56.721314 24279 raft_consensus.cc:729] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e005d8e1347a41578955f4106acf09e7, State: Initialized, Role: FOLLOWER
I20260926 20:50:56.721936 24279 consensus_queue.cc:260] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e005d8e1347a41578955f4106acf09e7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38967 } }
I20260926 20:50:56.722175 24279 raft_consensus.cc:388] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:50:56.722309 24279 raft_consensus.cc:482] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:50:56.722522 24279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:56.724788 24279 raft_consensus.cc:504] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e005d8e1347a41578955f4106acf09e7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38967 } }
I20260926 20:50:56.725374 24279 leader_election.cc:302] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e005d8e1347a41578955f4106acf09e7; no voters:
I20260926 20:50:56.726038 24279 leader_election.cc:288] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:50:56.726269 24284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:56.727361 24284 raft_consensus.cc:686] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [term 1 LEADER]: Becoming Leader. State: Replica: e005d8e1347a41578955f4106acf09e7, State: Running, Role: LEADER
I20260926 20:50:56.727888 24284 consensus_queue.cc:237] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e005d8e1347a41578955f4106acf09e7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38967 } }
I20260926 20:50:56.728289 24279 sys_catalog.cc:564] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:50:56.730310 24286 sys_catalog.cc:455] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e005d8e1347a41578955f4106acf09e7. Latest consensus state: current_term: 1 leader_uuid: "e005d8e1347a41578955f4106acf09e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e005d8e1347a41578955f4106acf09e7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38967 } } }
I20260926 20:50:56.730769 24286 sys_catalog.cc:458] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:56.730708 24285 sys_catalog.cc:455] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e005d8e1347a41578955f4106acf09e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e005d8e1347a41578955f4106acf09e7" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38967 } } }
I20260926 20:50:56.731019 24285 sys_catalog.cc:458] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7 [sys.catalog]: This master's current role is: LEADER
I20260926 20:50:56.733495 24290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:50:56.738667 24290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:50:56.763983 24290 catalog_manager.cc:1347] Generated new cluster ID: e3ee3f11facd48c99bc76fcf73f11fad
I20260926 20:50:56.764110 24290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:50:56.775913 24290 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:50:56.777806 24290 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:50:56.787912 24290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7: Generated new TSK 0
I20260926 20:50:56.789021 24290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:50:56.796371 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38967
--builtin_ntp_servers=127.5.78.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:50:57.144610 24303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:57.144938 24303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:57.145035 24303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:57.152679 24303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:57.152801 24303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:57.152917 24303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:50:57.162848 24303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:57.164163 24303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:57.165925 24303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:57.179344 24308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:57.179378 24309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:57.179411 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:57.180392 24303 server_base.cc:1034] running on GCE node
I20260926 20:50:57.181064 24303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:57.182291 24303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:57.183530 24303 hybrid_clock.cc:648] HybridClock initialized: now 1790455857183429 us; error 86 us; skew 500 ppm
I20260926 20:50:57.183949 24303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:57.186721 24303 webserver.cc:460] Webserver started at http://127.5.78.1:34065/ using document root <none> and password file <none>
I20260926 20:50:57.187386 24303 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:57.187470 24303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:57.187755 24303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:57.190711 24303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "120923ffc4e54e868a3b8887908ff1eb"
format_stamp: "Formatted at 2026-09-26 20:50:57 on dist-test-slave-z7j8"
server_key: "f9d8599bf42786dc7754c012df78976b"
server_key_iv: "5df72b47cf442e0cc240852461582aa3"
server_key_version: "encryptionkey@0"
I20260926 20:50:57.191462 24303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "120923ffc4e54e868a3b8887908ff1eb"
format_stamp: "Formatted at 2026-09-26 20:50:57 on dist-test-slave-z7j8"
server_key: "f9d8599bf42786dc7754c012df78976b"
server_key_iv: "5df72b47cf442e0cc240852461582aa3"
server_key_version: "encryptionkey@0"
I20260926 20:50:57.196768 24303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 20:50:57.200438 24317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:57.201821 24303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:50:57.202018 24303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "120923ffc4e54e868a3b8887908ff1eb"
format_stamp: "Formatted at 2026-09-26 20:50:57 on dist-test-slave-z7j8"
server_key: "f9d8599bf42786dc7754c012df78976b"
server_key_iv: "5df72b47cf442e0cc240852461582aa3"
server_key_version: "encryptionkey@0"
I20260926 20:50:57.202240 24303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:57.220968 24303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:57.221784 24303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:57.222056 24303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:57.223078 24303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:57.224570 24303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:57.224664 24303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:57.224794 24303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:57.224898 24303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:57.258627 24303 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:44179
I20260926 20:50:57.258677 24429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:44179 every 8 connection(s)
I20260926 20:50:57.260015 24303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:50:57.261080 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24303
I20260926 20:50:57.261281 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:50:57.261818 5432 external_mini_cluster.cc:1428] Setting key d3f273b1de0dacf65d7eea38f552bd41
I20260926 20:50:57.265548 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38967
--builtin_ntp_servers=127.5.78.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:50:57.273941 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:57.263072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:44699 (local address 127.5.78.62:38967)
0926 20:50:57.265345 (+ 2273us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:57.265360 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:57.265384 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:50:57.267144 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:57.267150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:57.267241 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:57.267369 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:57.268488 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.269211 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.270934 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.271343 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.271491 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:57.273179 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:57.273184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:57.273187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:57.273208 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:57.273248 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:57.273260 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:57.273341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:57.273590 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 20:50:57.273672 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2133,"thread_start_us":2035,"threads_started":1}
I20260926 20:50:57.275437 24433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:57.263726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38967 (local address 127.5.78.1:44699)
0926 20:50:57.264888 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:57.264935 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:57.266847 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:57.267626 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:57.267635 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:57.267732 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:57.268317 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:57.268334 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.269373 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.269383 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:57.270745 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:57.270757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.271588 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.271595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:57.271952 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:57.272776 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:50:57.272781 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:57.272847 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:50:57.272966 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:57.273366 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:57.273380 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:57.273391 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:50:57.273575 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 20:50:57.274018 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":379,"threads_started":1}
I20260926 20:50:57.277155 24430 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38967
I20260926 20:50:57.277477 24430 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:57.278280 24430 heartbeater.cc:511] Master 127.5.78.62:38967 requested a full tablet report, sending...
I20260926 20:50:57.280414 24244 ts_manager.cc:194] Registered new tserver with Master: 120923ffc4e54e868a3b8887908ff1eb (127.5.78.1:44179)
I20260926 20:50:57.281961 24244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.1:44699
W20260926 20:50:57.595181 24434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:57.595533 24434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:57.595691 24434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:57.603025 24434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:57.603175 24434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:57.603358 24434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:50:57.612735 24434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:57.614092 24434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:57.616106 24434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:57.629760 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:57.629822 24442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:57.629860 24440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:57.630669 24434 server_base.cc:1034] running on GCE node
I20260926 20:50:57.631299 24434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:57.632308 24434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:57.633523 24434 hybrid_clock.cc:648] HybridClock initialized: now 1790455857633481 us; error 39 us; skew 500 ppm
I20260926 20:50:57.633852 24434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:57.636253 24434 webserver.cc:460] Webserver started at http://127.5.78.2:45135/ using document root <none> and password file <none>
I20260926 20:50:57.636984 24434 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:57.637079 24434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:57.637435 24434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:57.640542 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "3eeb4805cca44c8aa8c6295b4f6e64af"
format_stamp: "Formatted at 2026-09-26 20:50:57 on dist-test-slave-z7j8"
server_key: "6af656e2e1ba005092c8978a4a2f1337"
server_key_iv: "64243e062c938c5b7b7d41ed71eb6e25"
server_key_version: "encryptionkey@0"
I20260926 20:50:57.641332 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "3eeb4805cca44c8aa8c6295b4f6e64af"
format_stamp: "Formatted at 2026-09-26 20:50:57 on dist-test-slave-z7j8"
server_key: "6af656e2e1ba005092c8978a4a2f1337"
server_key_iv: "64243e062c938c5b7b7d41ed71eb6e25"
server_key_version: "encryptionkey@0"
I20260926 20:50:57.646592 24434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:50:57.650058 24448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:57.651583 24434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:50:57.651753 24434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "3eeb4805cca44c8aa8c6295b4f6e64af"
format_stamp: "Formatted at 2026-09-26 20:50:57 on dist-test-slave-z7j8"
server_key: "6af656e2e1ba005092c8978a4a2f1337"
server_key_iv: "64243e062c938c5b7b7d41ed71eb6e25"
server_key_version: "encryptionkey@0"
I20260926 20:50:57.651973 24434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:57.666271 24434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:57.667141 24434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:57.667431 24434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:57.668386 24434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:57.669832 24434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:57.669946 24434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:57.670080 24434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:57.670171 24434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:57.703049 24434 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:34463
I20260926 20:50:57.703092 24560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:34463 every 8 connection(s)
I20260926 20:50:57.704625 24434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:50:57.711885 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24434
I20260926 20:50:57.712046 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:50:57.712589 5432 external_mini_cluster.cc:1428] Setting key 40dc7cc8cb902a7ab8e2bda06005391d
I20260926 20:50:57.715642 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38967
--builtin_ntp_servers=127.5.78.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:50:57.719123 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:57.707527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:57975 (local address 127.5.78.62:38967)
0926 20:50:57.707709 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:57.707715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:57.709283 (+ 1568us) server_negotiation.cc:407] Connection header received
0926 20:50:57.711085 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:57.711090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:57.711154 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:57.711248 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:57.712621 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.713471 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.715405 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.716090 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.716237 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:57.717850 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:57.717860 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:57.717905 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:57.717934 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:57.717977 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:57.717998 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:57.718090 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:57.718461 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 20:50:57.718619 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 20:50:57.720067 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:57.708070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38967 (local address 127.5.78.2:57975)
0926 20:50:57.709102 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:57.709153 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:57.710843 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:57.711466 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:57.711480 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:57.711593 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:57.712400 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:57.712425 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.713634 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.713639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:57.715242 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:57.715253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:57.716382 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:57.716387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:57.716593 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:57.717363 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:50:57.717366 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:57.717407 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:50:57.717487 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:57.718130 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:57.718144 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:57.718156 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:50:57.718438 (+ 282us) client_negotiation.cc:240] Negotiation successful
0926 20:50:57.718884 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":352,"threads_started":1}
I20260926 20:50:57.721549 24561 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38967
I20260926 20:50:57.721873 24561 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:57.722702 24561 heartbeater.cc:511] Master 127.5.78.62:38967 requested a full tablet report, sending...
I20260926 20:50:57.724323 24244 ts_manager.cc:194] Registered new tserver with Master: 3eeb4805cca44c8aa8c6295b4f6e64af (127.5.78.2:34463)
I20260926 20:50:57.725224 24244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.2:57975
W20260926 20:50:58.047946 24564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:50:58.048265 24564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:50:58.048416 24564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:50:58.056366 24564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:50:58.056491 24564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:50:58.056668 24564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:50:58.066692 24564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:38967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:50:58.068060 24564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:50:58.069880 24564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:50:58.083419 24572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:58.083499 24570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:50:58.083474 24569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:50:58.084427 24564 server_base.cc:1034] running on GCE node
I20260926 20:50:58.085141 24564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:50:58.086118 24564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:50:58.087349 24564 hybrid_clock.cc:648] HybridClock initialized: now 1790455858087266 us; error 70 us; skew 500 ppm
I20260926 20:50:58.087666 24564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:50:58.090032 24564 webserver.cc:460] Webserver started at http://127.5.78.3:35411/ using document root <none> and password file <none>
I20260926 20:50:58.090811 24564 fs_manager.cc:359] Metadata directory not provided
I20260926 20:50:58.090904 24564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:50:58.091173 24564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:50:58.094139 24564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "3a385200f6d745dd96603b7d63a5f888"
format_stamp: "Formatted at 2026-09-26 20:50:58 on dist-test-slave-z7j8"
server_key: "88a4833804e2dd947bc39cfadd777eef"
server_key_iv: "77b370f752bf5910f6b19fac52757ef4"
server_key_version: "encryptionkey@0"
I20260926 20:50:58.095041 24564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "3a385200f6d745dd96603b7d63a5f888"
format_stamp: "Formatted at 2026-09-26 20:50:58 on dist-test-slave-z7j8"
server_key: "88a4833804e2dd947bc39cfadd777eef"
server_key_iv: "77b370f752bf5910f6b19fac52757ef4"
server_key_version: "encryptionkey@0"
I20260926 20:50:58.100502 24564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:50:58.104212 24578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:58.105695 24564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:50:58.105871 24564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "3a385200f6d745dd96603b7d63a5f888"
format_stamp: "Formatted at 2026-09-26 20:50:58 on dist-test-slave-z7j8"
server_key: "88a4833804e2dd947bc39cfadd777eef"
server_key_iv: "77b370f752bf5910f6b19fac52757ef4"
server_key_version: "encryptionkey@0"
I20260926 20:50:58.106103 24564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:50:58.125977 24564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:50:58.126873 24564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:50:58.127147 24564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:50:58.128124 24564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:50:58.129737 24564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:50:58.129894 24564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:58.130018 24564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:50:58.130112 24564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:50:58.163836 24564 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:39793
I20260926 20:50:58.163923 24690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:39793 every 8 connection(s)
I20260926 20:50:58.165099 24564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:50:58.173157 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24564
I20260926 20:50:58.173346 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:50:58.173777 5432 external_mini_cluster.cc:1428] Setting key a28ea9122ec8f7be51e9b6d0f75d54c5
I20260926 20:50:58.178006 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.167638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:46007 (local address 127.5.78.62:38967)
0926 20:50:58.167816 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.167820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.169192 (+ 1372us) server_negotiation.cc:407] Connection header received
0926 20:50:58.171028 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.171034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.171127 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.171239 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:58.172381 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.173151 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.175051 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.175693 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.175843 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.177244 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:58.177250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:58.177253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:58.177273 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:58.177307 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:58.177321 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:58.177416 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.177686 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.177767 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:50:58.179162 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.168168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:38967 (local address 127.5.78.3:46007)
0926 20:50:58.169036 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:58.169077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:58.170757 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:58.171500 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:58.171509 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:58.171600 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:50:58.172194 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.172212 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.173323 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.173333 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.174846 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.174860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.175943 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.175950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.176183 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.176886 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:50:58.176889 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:50:58.176930 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:50:58.177033 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:50:58.177446 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:50:58.177456 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:50:58.177462 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 20:50:58.177642 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 20:50:58.177953 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":267,"threads_started":1}
I20260926 20:50:58.180538 24691 heartbeater.cc:347] Connected to a master server at 127.5.78.62:38967
I20260926 20:50:58.180835 24691 heartbeater.cc:464] Registering TS with master...
I20260926 20:50:58.181595 24691 heartbeater.cc:511] Master 127.5.78.62:38967 requested a full tablet report, sending...
I20260926 20:50:58.183004 24244 ts_manager.cc:194] Registered new tserver with Master: 3a385200f6d745dd96603b7d63a5f888 (127.5.78.3:39793)
I20260926 20:50:58.183800 24244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.3:46007
I20260926 20:50:58.189904 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:50:58.199894 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.194080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.5.78.62:38967)
0926 20:50:58.194270 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.194275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.194714 (+ 439us) server_negotiation.cc:407] Connection header received
0926 20:50:58.194856 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.194861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.194938 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.195033 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:58.195466 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.196160 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.197545 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.197921 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.198027 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.199230 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:58.199236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:58.199239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:58.199258 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:58.199292 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:58.199305 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:58.199454 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.199608 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.199657 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:50:58.204108 24244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:50:58.211417 24244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:50:58.234215 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.223721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:34463 (local address 127.0.0.1:47382)
0926 20:50:58.225107 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:58.225162 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:58.225430 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:58.226500 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:58.226508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:58.226536 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:58.227074 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.227092 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.229142 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.229147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.231337 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.231358 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.233206 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.233212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.233442 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.233509 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 20:50:58.233649 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 20:50:58.233834 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"mutex_wait_us":51,"thread_start_us":2015,"threads_started":1}
I20260926 20:50:58.234310 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.223949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.5.78.2:34463)
0926 20:50:58.224588 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.224595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.225273 (+ 678us) server_negotiation.cc:407] Connection header received
0926 20:50:58.225673 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.225686 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.225970 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.226178 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:58.227300 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.228905 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.231542 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.232689 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.232836 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.233086 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.233807 (+ 721us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.233967 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":363,"threads_started":1}
I20260926 20:50:58.235474 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.224615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.5.78.3:39793)
0926 20:50:58.225375 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.225383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.226289 (+ 906us) server_negotiation.cc:407] Connection header received
0926 20:50:58.226391 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.226400 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.226681 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.226867 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:58.227933 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.229346 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.231979 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.233414 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.233540 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.233805 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.234899 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.235081 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":1079,"threads_started":1}
I20260926 20:50:58.235639 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.226744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44179 (local address 127.0.0.1:35170)
0926 20:50:58.227518 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:58.227540 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:58.227659 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:58.228684 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:58.228688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:58.228711 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:58.229017 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.229026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.230992 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.230997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.233039 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.233053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.234915 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.234921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.235082 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.235111 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 20:50:58.235208 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:50:58.235329 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":638,"threads_started":1}
I20260926 20:50:58.236018 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.224002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:39793 (local address 127.0.0.1:54080)
0926 20:50:58.226019 (+ 2017us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:58.226037 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:58.226162 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:58.227131 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:58.227139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:58.227165 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:58.227650 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.227668 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.229544 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.229549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.231778 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.231796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.233926 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.233934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.234182 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.234224 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:50:58.235442 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0926 20:50:58.235556 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1872,"mutex_wait_us":78,"thread_start_us":926,"threads_started":1}
I20260926 20:50:58.236672 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.227200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.5.78.1:44179)
0926 20:50:58.228150 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.228158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.228184 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:50:58.228265 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.228272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.228488 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.228664 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:58.229189 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.230850 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.233271 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.234726 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.234869 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.235083 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.235265 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.236342 (+ 1077us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":667,"threads_started":1}
I20260926 20:50:58.238384 24496 tablet_service.cc:1467] Processing CreateTablet for tablet 37ebca2d62714c378e966992e5106e1c (DEFAULT_TABLE table=test-table [id=cdb99cf5c98f4c8ba83cb121ec6f7271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:58.239921 24496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ebca2d62714c378e966992e5106e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:58.239698 24626 tablet_service.cc:1467] Processing CreateTablet for tablet 37ebca2d62714c378e966992e5106e1c (DEFAULT_TABLE table=test-table [id=cdb99cf5c98f4c8ba83cb121ec6f7271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:58.239940 24365 tablet_service.cc:1467] Processing CreateTablet for tablet 37ebca2d62714c378e966992e5106e1c (DEFAULT_TABLE table=test-table [id=cdb99cf5c98f4c8ba83cb121ec6f7271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:50:58.241088 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ebca2d62714c378e966992e5106e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:58.241123 24626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ebca2d62714c378e966992e5106e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:50:58.253110 24709 tablet_bootstrap.cc:492] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Bootstrap starting.
I20260926 20:50:58.256498 24709 tablet_bootstrap.cc:654] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:58.257761 24709 log.cc:824] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:58.259605 24711 tablet_bootstrap.cc:492] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Bootstrap starting.
I20260926 20:50:58.264508 24710 tablet_bootstrap.cc:492] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Bootstrap starting.
I20260926 20:50:58.267073 24711 tablet_bootstrap.cc:654] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:58.268023 24710 tablet_bootstrap.cc:654] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Neither blocks nor log segments found. Creating new log.
I20260926 20:50:58.268179 24709 tablet_bootstrap.cc:492] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: No bootstrap required, opened a new log
I20260926 20:50:58.268405 24711 log.cc:824] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:58.268478 24709 ts_tablet_manager.cc:1397] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260926 20:50:58.269240 24710 log.cc:824] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Log is configured to *not* fsync() on all Append() calls
I20260926 20:50:58.277799 24711 tablet_bootstrap.cc:492] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: No bootstrap required, opened a new log
I20260926 20:50:58.278185 24711 ts_tablet_manager.cc:1397] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.010s
I20260926 20:50:58.278256 24710 tablet_bootstrap.cc:492] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: No bootstrap required, opened a new log
I20260926 20:50:58.278527 24710 ts_tablet_manager.cc:1397] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Time spent bootstrapping tablet: real 0.014s user 0.014s sys 0.000s
I20260926 20:50:58.282193 24709 raft_consensus.cc:348] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.282692 24709 raft_consensus.cc:374] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:58.282800 24709 raft_consensus.cc:729] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 120923ffc4e54e868a3b8887908ff1eb, State: Initialized, Role: FOLLOWER
I20260926 20:50:58.283430 24709 consensus_queue.cc:260] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.284593 24430 heartbeater.cc:503] Master 127.5.78.62:38967 was elected leader, sending a full tablet report...
I20260926 20:50:58.285264 24709 ts_tablet_manager.cc:1428] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
I20260926 20:50:58.287806 24710 raft_consensus.cc:348] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.288177 24710 raft_consensus.cc:374] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:58.288300 24710 raft_consensus.cc:729] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eeb4805cca44c8aa8c6295b4f6e64af, State: Initialized, Role: FOLLOWER
I20260926 20:50:58.288183 24711 raft_consensus.cc:348] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.288637 24711 raft_consensus.cc:374] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:50:58.288733 24711 raft_consensus.cc:729] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a385200f6d745dd96603b7d63a5f888, State: Initialized, Role: FOLLOWER
I20260926 20:50:58.288904 24710 consensus_queue.cc:260] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.289319 24711 consensus_queue.cc:260] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.291105 24691 heartbeater.cc:503] Master 127.5.78.62:38967 was elected leader, sending a full tablet report...
I20260926 20:50:58.291548 24561 heartbeater.cc:503] Master 127.5.78.62:38967 was elected leader, sending a full tablet report...
I20260926 20:50:58.291965 24711 ts_tablet_manager.cc:1428] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260926 20:50:58.292172 24710 ts_tablet_manager.cc:1428] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
W20260926 20:50:58.416837 24692 tablet.cc:2369] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:50:58.456734 24562 tablet.cc:2369] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:58.502825 24716 raft_consensus.cc:482] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:50:58.503193 24716 raft_consensus.cc:504] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.505002 24716 leader_election.cc:288] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a385200f6d745dd96603b7d63a5f888 (127.5.78.3:39793), 120923ffc4e54e868a3b8887908ff1eb (127.5.78.1:44179)
I20260926 20:50:58.512320 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.505557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:59627 (local address 127.5.78.3:39793)
0926 20:50:58.505776 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.505784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.506602 (+ 818us) server_negotiation.cc:407] Connection header received
0926 20:50:58.506802 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.506806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.506922 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.507019 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:58.507654 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.508350 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.510552 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.511410 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.511575 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.511671 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.511949 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.512073 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:50:58.512362 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.505560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:41989 (local address 127.5.78.1:44179)
0926 20:50:58.505776 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.505784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.506392 (+ 608us) server_negotiation.cc:407] Connection header received
0926 20:50:58.506650 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.506656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.506770 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.506863 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:50:58.507537 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.508345 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.510543 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.511387 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.511482 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.511607 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.511979 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.512108 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 20:50:58.512583 24431 tablet.cc:2369] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:50:58.512724 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.505553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:39793 (local address 127.5.78.2:59627)
0926 20:50:58.506487 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:58.506502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:58.506647 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:58.507059 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:58.507063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:58.507094 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:58.507493 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.507506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.508490 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.508496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.510283 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.510294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.511630 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.511638 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.511788 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.511816 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 20:50:58.511892 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 20:50:58.512051 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":108,"thread_start_us":525,"threads_started":1}
I20260926 20:50:58.512985 24718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.505281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:44179 (local address 127.5.78.2:41989)
0926 20:50:58.506258 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0926 20:50:58.506278 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:50:58.506499 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:50:58.506893 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:50:58.506898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:50:58.506921 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:50:58.507381 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.507390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.508477 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.508485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.510267 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:50:58.510279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.511586 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.511592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:50:58.511797 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.511820 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:50:58.511931 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 20:50:58.512072 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":99,"thread_start_us":627,"threads_started":1}
I20260926 20:50:58.513918 24646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ebca2d62714c378e966992e5106e1c" candidate_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a385200f6d745dd96603b7d63a5f888" is_pre_election: true
I20260926 20:50:58.513913 24385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ebca2d62714c378e966992e5106e1c" candidate_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "120923ffc4e54e868a3b8887908ff1eb" is_pre_election: true
I20260926 20:50:58.514452 24385 raft_consensus.cc:2454] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eeb4805cca44c8aa8c6295b4f6e64af in term 0.
I20260926 20:50:58.514451 24646 raft_consensus.cc:2454] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eeb4805cca44c8aa8c6295b4f6e64af in term 0.
I20260926 20:50:58.515122 24449 leader_election.cc:302] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a385200f6d745dd96603b7d63a5f888, 3eeb4805cca44c8aa8c6295b4f6e64af; no voters:
I20260926 20:50:58.515512 24716 raft_consensus.cc:2781] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:50:58.515661 24716 raft_consensus.cc:482] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:50:58.515781 24716 raft_consensus.cc:3037] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:58.517951 24716 raft_consensus.cc:504] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.518668 24716 leader_election.cc:288] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [CANDIDATE]: Term 1 election: Requested vote from peers 3a385200f6d745dd96603b7d63a5f888 (127.5.78.3:39793), 120923ffc4e54e868a3b8887908ff1eb (127.5.78.1:44179)
I20260926 20:50:58.519109 24646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ebca2d62714c378e966992e5106e1c" candidate_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a385200f6d745dd96603b7d63a5f888"
I20260926 20:50:58.519135 24385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ebca2d62714c378e966992e5106e1c" candidate_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "120923ffc4e54e868a3b8887908ff1eb"
I20260926 20:50:58.519313 24646 raft_consensus.cc:3037] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:58.519323 24385 raft_consensus.cc:3037] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:50:58.521461 24646 raft_consensus.cc:2454] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eeb4805cca44c8aa8c6295b4f6e64af in term 1.
I20260926 20:50:58.521467 24385 raft_consensus.cc:2454] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eeb4805cca44c8aa8c6295b4f6e64af in term 1.
I20260926 20:50:58.521889 24449 leader_election.cc:302] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a385200f6d745dd96603b7d63a5f888, 3eeb4805cca44c8aa8c6295b4f6e64af; no voters:
I20260926 20:50:58.522202 24716 raft_consensus.cc:2781] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:50:58.522668 24716 raft_consensus.cc:686] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 1 LEADER]: Becoming Leader. State: Replica: 3eeb4805cca44c8aa8c6295b4f6e64af, State: Running, Role: LEADER
I20260926 20:50:58.523156 24716 consensus_queue.cc:237] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } }
I20260926 20:50:58.527741 24242 catalog_manager.cc:5495] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eeb4805cca44c8aa8c6295b4f6e64af (127.5.78.2). New cstate: current_term: 1 leader_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eeb4805cca44c8aa8c6295b4f6e64af" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 } health_report { overall_health: UNKNOWN } } }
I20260926 20:50:58.550549 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.544846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.5.78.2:34463)
0926 20:50:58.545018 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.545022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.545183 (+ 161us) server_negotiation.cc:407] Connection header received
0926 20:50:58.545360 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.545364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.545448 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.545568 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:50:58.545961 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.546740 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.548311 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.548739 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.548860 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.549411 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:50:58.549906 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:50:58.550038 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.550229 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.550268 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 20:50:58.559146 24385 raft_consensus.cc:1261] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 1 FOLLOWER]: Refusing update from remote peer 3eeb4805cca44c8aa8c6295b4f6e64af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:58.559329 24646 raft_consensus.cc:1261] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 1 FOLLOWER]: Refusing update from remote peer 3eeb4805cca44c8aa8c6295b4f6e64af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:50:58.560173 24720 consensus_queue.cc:1035] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [LEADER]: Connected to new peer: Peer: permanent_uuid: "120923ffc4e54e868a3b8887908ff1eb" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 44179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:58.560526 24716 consensus_queue.cc:1035] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a385200f6d745dd96603b7d63a5f888" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:50:58.572932 24725 mvcc.cc:205] Tried to move back new op lower bound from 7333707196647714816 to 7333707196514033664. Current Snapshot: MvccSnapshot[applied={T|T < 7333707196647714816}]
I20260926 20:50:58.585187 24244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:58.585516 24244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:50:58.589834 24244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e005d8e1347a41578955f4106acf09e7: Sending DeleteTablet for 3 replicas of tablet 37ebca2d62714c378e966992e5106e1c
I20260926 20:50:58.591051 24496 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ebca2d62714c378e966992e5106e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:58 UTC) from {username='slave'} at 127.0.0.1:47382
I20260926 20:50:58.591145 24365 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ebca2d62714c378e966992e5106e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:58 UTC) from {username='slave'} at 127.0.0.1:35170
I20260926 20:50:58.591195 24626 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ebca2d62714c378e966992e5106e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:50:58 UTC) from {username='slave'} at 127.0.0.1:54080
I20260926 20:50:58.592260 24734 tablet_replica.cc:330] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: stopping tablet replica
I20260926 20:50:58.592725 24734 raft_consensus.cc:2229] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:50:58.592995 24733 tablet_replica.cc:330] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: stopping tablet replica
I20260926 20:50:58.593412 24733 raft_consensus.cc:2229] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:58.593423 24734 raft_consensus.cc:2258] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:58.593950 24733 raft_consensus.cc:2258] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:58.594551 24735 tablet_replica.cc:330] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: stopping tablet replica
I20260926 20:50:58.594683 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24303
I20260926 20:50:58.594954 24735 raft_consensus.cc:2229] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:50:58.595439 24735 raft_consensus.cc:2258] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:50:58.595835 24734 ts_tablet_manager.cc:1905] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:58.597524 24733 ts_tablet_manager.cc:1905] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:58.604156 24734 ts_tablet_manager.cc:1918] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:58.604350 24734 log.cc:1196] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/37ebca2d62714c378e966992e5106e1c
I20260926 20:50:58.605044 24734 ts_tablet_manager.cc:1939] T 37ebca2d62714c378e966992e5106e1c P 3eeb4805cca44c8aa8c6295b4f6e64af: Deleting consensus metadata
I20260926 20:50:58.607372 24733 ts_tablet_manager.cc:1918] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:58.607372 24231 catalog_manager.cc:4841] TS 3eeb4805cca44c8aa8c6295b4f6e64af (127.5.78.2:34463): tablet 37ebca2d62714c378e966992e5106e1c (table test-table [id=cdb99cf5c98f4c8ba83cb121ec6f7271]) successfully deleted
I20260926 20:50:58.607550 24733 log.cc:1196] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/37ebca2d62714c378e966992e5106e1c
I20260926 20:50:58.608079 24733 ts_tablet_manager.cc:1939] T 37ebca2d62714c378e966992e5106e1c P 120923ffc4e54e868a3b8887908ff1eb: Deleting consensus metadata
I20260926 20:50:58.608600 24735 ts_tablet_manager.cc:1905] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:50:58.610059 24229 catalog_manager.cc:4841] TS 120923ffc4e54e868a3b8887908ff1eb (127.5.78.1:44179): tablet 37ebca2d62714c378e966992e5106e1c (table test-table [id=cdb99cf5c98f4c8ba83cb121ec6f7271]) successfully deleted
I20260926 20:50:58.615276 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.596898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35184 (local address 127.5.78.1:44179)
0926 20:50:58.597076 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.597082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.608316 (+ 11234us) server_negotiation.cc:407] Connection header received
0926 20:50:58.608622 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.608628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.608730 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.608808 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:58.609562 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.610677 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.612108 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.612527 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.612668 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.613812 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:58.613820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:58.613833 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:58.614499 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:58.614549 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:58.614568 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:58.614689 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.614834 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.614942 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:50:58.616169 24425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:58.616343 24735 ts_tablet_manager.cc:1918] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:50:58.616524 24735 log.cc:1196] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/37ebca2d62714c378e966992e5106e1c
I20260926 20:50:58.617105 24735 ts_tablet_manager.cc:1939] T 37ebca2d62714c378e966992e5106e1c P 3a385200f6d745dd96603b7d63a5f888: Deleting consensus metadata
I20260926 20:50:58.619709 24228 catalog_manager.cc:4841] TS 3a385200f6d745dd96603b7d63a5f888 (127.5.78.3:39793): tablet 37ebca2d62714c378e966992e5106e1c (table test-table [id=cdb99cf5c98f4c8ba83cb121ec6f7271]) successfully deleted
I20260926 20:50:58.746990 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24303
I20260926 20:50:58.772435 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24434
I20260926 20:50:58.780743 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.773350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.5.78.2:34463)
0926 20:50:58.773524 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.773528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.773592 (+ 64us) server_negotiation.cc:407] Connection header received
0926 20:50:58.773782 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.773786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.773875 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.773963 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:58.774514 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.775293 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.777363 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.777790 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.777892 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.779272 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:58.779280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:58.779294 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:58.779909 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:58.779958 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:58.779978 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:58.780058 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.780240 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.780317 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:50:58.781481 24556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:58.893824 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24434
I20260926 20:50:58.912451 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24564
I20260926 20:50:58.919672 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:50:58.913342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.5.78.3:39793)
0926 20:50:58.913526 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:50:58.913534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:50:58.913560 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:50:58.913799 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:50:58.913804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:50:58.913895 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:50:58.914004 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:50:58.914452 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.915166 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.916470 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:50:58.916873 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:50:58.917006 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:50:58.918110 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:50:58.918120 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:50:58.918134 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:50:58.918891 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:50:58.918941 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:50:58.918960 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:50:58.919105 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 20:50:58.919238 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 20:50:58.919345 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:50:58.920357 24686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:59.010066 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24564
I20260926 20:50:59.026676 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24213
I20260926 20:50:59.027840 24274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:50:59.143551 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24213
2026-09-26T20:50:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2987 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:50:59 dist-test-slave-z7j8 krb5kdc[24743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:50:59 dist-test-slave-z7j8 krb5kdc[24743](info): set up 2 sockets
Sep 26 20:50:59 dist-test-slave-z7j8 krb5kdc[24743](info): commencing operation
krb5kdc: starting...
W20260926 20:51:00.254720 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:00 dist-test-slave-z7j8 krb5kdc[24743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:00.521392 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:39327
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:39327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:00.862613 24756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:00.862995 24756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:00.863148 24756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:00.870707 24756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:00.870864 24756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:00.870973 24756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:00.871081 24756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:00.871156 24756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:00.881295 24756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:39327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:39327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:00.882707 24756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:00.884035 24756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:00.895845 24764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:00.895941 24761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:00.895915 24762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:00.899936 24756 server_base.cc:1034] running on GCE node
I20260926 20:51:00.900734 24756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:00.902263 24756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:00.903496 24756 hybrid_clock.cc:648] HybridClock initialized: now 1790455860903456 us; error 55 us; skew 500 ppm
I20260926 20:51:00.903852 24756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:00 dist-test-slave-z7j8 krb5kdc[24743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455860, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:00.907160 24756 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:00.908874 24756 webserver.cc:460] Webserver started at http://127.5.78.62:42069/ using document root <none> and password file <none>
I20260926 20:51:00.909624 24756 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:00.909777 24756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:00.910089 24756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:00.913327 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "f521cab210eb4599a61fb2d5c23f6cf4"
format_stamp: "Formatted at 2026-09-26 20:51:00 on dist-test-slave-z7j8"
server_key: "2a6d1ff3c7d92f7ea94472e2173ca50c"
server_key_iv: "defacd8a6ae1734c675e7f50bbedb511"
server_key_version: "encryptionkey@0"
I20260926 20:51:00.914160 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "f521cab210eb4599a61fb2d5c23f6cf4"
format_stamp: "Formatted at 2026-09-26 20:51:00 on dist-test-slave-z7j8"
server_key: "2a6d1ff3c7d92f7ea94472e2173ca50c"
server_key_iv: "defacd8a6ae1734c675e7f50bbedb511"
server_key_version: "encryptionkey@0"
I20260926 20:51:00.919878 24756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:51:00.923815 24771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:00.925347 24756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:00.925555 24756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "f521cab210eb4599a61fb2d5c23f6cf4"
format_stamp: "Formatted at 2026-09-26 20:51:00 on dist-test-slave-z7j8"
server_key: "2a6d1ff3c7d92f7ea94472e2173ca50c"
server_key_iv: "defacd8a6ae1734c675e7f50bbedb511"
server_key_version: "encryptionkey@0"
I20260926 20:51:00.925771 24756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:00.948951 24756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:00.953171 24756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:00.953470 24756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:00.972702 24756 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:39327
I20260926 20:51:00.972759 24822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:39327 every 8 connection(s)
I20260926 20:51:00.974061 24756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:00.977643 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24756
I20260926 20:51:00.977826 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:00.977910 24823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:00.978386 5432 external_mini_cluster.cc:1428] Setting key 004735d9edf30554836e58c83d168f26
I20260926 20:51:00.993680 24823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4: Bootstrap starting.
Sep 26 20:51:00 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:00.998081 24823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:00.999636 24823 log.cc:824] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:01.003412 24823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4: No bootstrap required, opened a new log
I20260926 20:51:01.004503 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:00.980221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.5.78.62:39327)
0926 20:51:00.981046 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:00.981058 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:00.981099 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:51:00.982858 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:00.982881 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:00.983282 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:00.983611 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:00.985618 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:00.987023 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:00.988949 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:00.989658 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:00.989796 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:00.995159 (+ 5363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:00.995181 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:00.995201 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:00.995269 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:00.999495 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:01.000295 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:01.000310 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:01.000318 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:01.000556 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:01.001053 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:01.001056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:01.001058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:01.001817 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:01.002127 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:01.002566 (+ 439us) server_negotiation.cc:299] Negotiation successful
0926 20:51:01.003012 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":389,"threads_started":1}
I20260926 20:51:01.012439 24823 raft_consensus.cc:348] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39327 } }
I20260926 20:51:01.012852 24823 raft_consensus.cc:374] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:01.012938 24823 raft_consensus.cc:729] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f521cab210eb4599a61fb2d5c23f6cf4, State: Initialized, Role: FOLLOWER
I20260926 20:51:01.013506 24823 consensus_queue.cc:260] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39327 } }
I20260926 20:51:01.013733 24823 raft_consensus.cc:388] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:01.013845 24823 raft_consensus.cc:482] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:01.013978 24823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:01.016333 24823 raft_consensus.cc:504] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39327 } }
I20260926 20:51:01.016994 24823 leader_election.cc:302] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f521cab210eb4599a61fb2d5c23f6cf4; no voters:
I20260926 20:51:01.017611 24823 leader_election.cc:288] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:01.017783 24828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:01.018546 24828 raft_consensus.cc:686] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [term 1 LEADER]: Becoming Leader. State: Replica: f521cab210eb4599a61fb2d5c23f6cf4, State: Running, Role: LEADER
I20260926 20:51:01.019176 24828 consensus_queue.cc:237] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39327 } }
I20260926 20:51:01.019943 24823 sys_catalog.cc:564] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:01.022320 24830 sys_catalog.cc:455] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f521cab210eb4599a61fb2d5c23f6cf4. Latest consensus state: current_term: 1 leader_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39327 } } }
I20260926 20:51:01.022662 24830 sys_catalog.cc:458] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:01.022251 24829 sys_catalog.cc:455] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f521cab210eb4599a61fb2d5c23f6cf4" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 39327 } } }
I20260926 20:51:01.023357 24829 sys_catalog.cc:458] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:01.027343 24834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:01.034334 24834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:01.044131 24834 catalog_manager.cc:1347] Generated new cluster ID: c6befa6a3c4046a7ba5e0e0987308422
I20260926 20:51:01.044267 24834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:01.063833 24834 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:01.066073 24834 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:01.077008 24834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4: Generated new TSK 0
I20260926 20:51:01.077908 24834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.78.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
Entry for principal kudu/127.5.78.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab.
I20260926 20:51:01.146106 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39327
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:01.492450 24849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:01.492733 24849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:01.492892 24849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:01.500391 24849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:01.500514 24849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:01.500689 24849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:51:01.511474 24849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.1.keytab
--principal=kudu/127.5.78.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:01.512842 24849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:01.514189 24849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:01.528007 24854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:01.528252 24857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:01.527940 24855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:01.529139 24849 server_base.cc:1034] running on GCE node
I20260926 20:51:01.529906 24849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:01.531055 24849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:01.532346 24849 hybrid_clock.cc:648] HybridClock initialized: now 1790455861532291 us; error 37 us; skew 500 ppm
I20260926 20:51:01.532728 24849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:01 dist-test-slave-z7j8 krb5kdc[24743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455861, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:01.536401 24849 init.cc:377] Logged in from keytab as kudu/127.5.78.1@KRBTEST.COM (short username kudu)
I20260926 20:51:01.538038 24849 webserver.cc:460] Webserver started at http://127.5.78.1:38321/ using document root <none> and password file <none>
I20260926 20:51:01.538882 24849 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:01.539050 24849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:01.539377 24849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:01.542644 24849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "db451fd1568345d1a9019234ea606fef"
format_stamp: "Formatted at 2026-09-26 20:51:01 on dist-test-slave-z7j8"
server_key: "e30a0dad3c7c0f1c699ba9c0b0854336"
server_key_iv: "b9ca90b569ad96c18e2400fb796ae697"
server_key_version: "encryptionkey@0"
I20260926 20:51:01.543550 24849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "db451fd1568345d1a9019234ea606fef"
format_stamp: "Formatted at 2026-09-26 20:51:01 on dist-test-slave-z7j8"
server_key: "e30a0dad3c7c0f1c699ba9c0b0854336"
server_key_iv: "b9ca90b569ad96c18e2400fb796ae697"
server_key_version: "encryptionkey@0"
I20260926 20:51:01.549346 24849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:51:01.553551 24864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:01.555338 24849 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:51:01.555596 24849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "db451fd1568345d1a9019234ea606fef"
format_stamp: "Formatted at 2026-09-26 20:51:01 on dist-test-slave-z7j8"
server_key: "e30a0dad3c7c0f1c699ba9c0b0854336"
server_key_iv: "b9ca90b569ad96c18e2400fb796ae697"
server_key_version: "encryptionkey@0"
I20260926 20:51:01.555861 24849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:01.577767 24849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:01.582732 24849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:01.583061 24849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:01.584146 24849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:51:01.585640 24849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:51:01.585749 24849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:01.585904 24849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:51:01.585978 24849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:01.620314 24849 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:33639
I20260926 20:51:01.620406 24976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:33639 every 8 connection(s)
I20260926 20:51:01.621774 24849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:51:01.631943 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24849
I20260926 20:51:01.632128 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:51:01.632555 5432 external_mini_cluster.cc:1428] Setting key c92027871656253643b183ea9aaf691c
Sep 26 20:51:01 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455861, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.1@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:01.646555 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:01.624596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:44895 (local address 127.5.78.62:39327)
0926 20:51:01.625038 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:01.625042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:01.626145 (+ 1103us) server_negotiation.cc:407] Connection header received
0926 20:51:01.628257 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:01.628262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:01.628353 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:01.628485 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:01.630129 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:01.630997 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:01.632729 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:01.633142 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:01.633289 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:01.640446 (+ 7157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:01.640451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:01.640454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:01.640498 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:01.643236 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:01.644061 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:01.644070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:01.644073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:01.644209 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:01.644697 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:01.644703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:01.644706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:01.644987 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:01.645162 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:01.645909 (+ 747us) server_negotiation.cc:299] Negotiation successful
0926 20:51:01.646149 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":241,"threads_started":1}
I20260926 20:51:01.647585 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:01.625253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39327 (local address 127.5.78.1:44895)
0926 20:51:01.625980 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:01.626020 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:01.627899 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:01.628733 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:01.628743 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:01.629363 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:01.629943 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:01.629961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:01.631167 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:01.631171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:01.632565 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:01.632578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:01.633391 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:01.633400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:01.633711 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:01.637730 (+ 4019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:01.637761 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:01.640157 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:01.643419 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:01.643429 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:01.643449 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:01.643911 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:01.644332 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:01.644338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:01.644342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:01.644557 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:01.645122 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:01.645131 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:01.645518 (+ 387us) client_negotiation.cc:769] Sending connection context
0926 20:51:01.645812 (+ 294us) client_negotiation.cc:240] Negotiation successful
0926 20:51:01.646385 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":289,"threads_started":1}
I20260926 20:51:01.649313 24977 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39327
I20260926 20:51:01.649654 24977 heartbeater.cc:464] Registering TS with master...
I20260926 20:51:01.650590 24977 heartbeater.cc:511] Master 127.5.78.62:39327 requested a full tablet report, sending...
I20260926 20:51:01.653200 24788 ts_manager.cc:194] Registered new tserver with Master: db451fd1568345d1a9019234ea606fef (127.5.78.1:33639)
I20260926 20:51:01.655592 24788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.1@KRBTEST.COM'} at 127.5.78.1:44895
WARNING: no policy specified for kudu/127.5.78.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
Entry for principal kudu/127.5.78.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab.
I20260926 20:51:01.690081 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39327
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:02.038041 24983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:02.038329 24983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:02.038504 24983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:02.045912 24983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:02.046032 24983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:02.046216 24983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:51:02.056252 24983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.2.keytab
--principal=kudu/127.5.78.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:02.057641 24983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:02.059219 24983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:02.074107 24991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:02.074152 24989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:02.074043 24988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:02.075042 24983 server_base.cc:1034] running on GCE node
I20260926 20:51:02.075637 24983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:02.076627 24983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:02.077852 24983 hybrid_clock.cc:648] HybridClock initialized: now 1790455862077794 us; error 52 us; skew 500 ppm
I20260926 20:51:02.078230 24983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:02 dist-test-slave-z7j8 krb5kdc[24743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455862, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:02.081629 24983 init.cc:377] Logged in from keytab as kudu/127.5.78.2@KRBTEST.COM (short username kudu)
I20260926 20:51:02.083360 24983 webserver.cc:460] Webserver started at http://127.5.78.2:35811/ using document root <none> and password file <none>
I20260926 20:51:02.084131 24983 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:02.084263 24983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:02.084594 24983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:02.087666 24983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "b5617dd418834d66bbae7ddf37b71421"
format_stamp: "Formatted at 2026-09-26 20:51:02 on dist-test-slave-z7j8"
server_key: "40ab92463d5ab020bef389ebf599b8ba"
server_key_iv: "975c427a624154131c2bb1328ff72bfa"
server_key_version: "encryptionkey@0"
I20260926 20:51:02.088426 24983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "b5617dd418834d66bbae7ddf37b71421"
format_stamp: "Formatted at 2026-09-26 20:51:02 on dist-test-slave-z7j8"
server_key: "40ab92463d5ab020bef389ebf599b8ba"
server_key_iv: "975c427a624154131c2bb1328ff72bfa"
server_key_version: "encryptionkey@0"
I20260926 20:51:02.094003 24983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:51:02.097765 24998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:02.099298 24983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:02.099495 24983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "b5617dd418834d66bbae7ddf37b71421"
format_stamp: "Formatted at 2026-09-26 20:51:02 on dist-test-slave-z7j8"
server_key: "40ab92463d5ab020bef389ebf599b8ba"
server_key_iv: "975c427a624154131c2bb1328ff72bfa"
server_key_version: "encryptionkey@0"
I20260926 20:51:02.099732 24983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:02.116706 24983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:02.121865 24983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:02.122205 24983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:02.123275 24983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:51:02.124720 24983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:51:02.124835 24983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:02.125110 24983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:51:02.125245 24983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:02.160558 24983 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:44553
I20260926 20:51:02.160607 25110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:44553 every 8 connection(s)
I20260926 20:51:02.161933 24983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:51:02.166366 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 24983
I20260926 20:51:02.166576 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:51:02.167062 5432 external_mini_cluster.cc:1428] Setting key 6a81b86c17709a0a94d9a3c1dfb39290
Sep 26 20:51:02 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455862, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.2@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:02.184412 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.164786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:52817 (local address 127.5.78.62:39327)
0926 20:51:02.165358 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:02.165365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:02.166464 (+ 1099us) server_negotiation.cc:407] Connection header received
0926 20:51:02.168832 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:02.168838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:02.168926 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:02.169027 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:02.170758 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.171482 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.173139 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.173591 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.173740 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.177677 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:02.177683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:02.177686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:02.177735 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:02.180534 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:02.181613 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:02.181618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:02.181620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:02.181771 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:02.182358 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:02.182363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:02.182365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:02.182784 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:02.182923 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:02.183920 (+ 997us) server_negotiation.cc:299] Negotiation successful
0926 20:51:02.184118 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":241,"threads_started":1}
I20260926 20:51:02.186033 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.165443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39327 (local address 127.5.78.2:52817)
0926 20:51:02.166272 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:02.166311 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:02.168483 (+ 2172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:02.169346 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:02.169355 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:02.169890 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:02.170567 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.170590 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.171681 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.171685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.172967 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.172980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.173840 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.173849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.174126 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.174963 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:02.174989 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:02.177406 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:02.180695 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:02.180708 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:02.180730 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:02.181420 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:02.181894 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:02.181901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:02.181908 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:02.182188 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:02.182935 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:02.182946 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:02.183461 (+ 515us) client_negotiation.cc:769] Sending connection context
0926 20:51:02.183799 (+ 338us) client_negotiation.cc:240] Negotiation successful
0926 20:51:02.184459 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":308,"threads_started":1}
I20260926 20:51:02.187799 25111 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39327
I20260926 20:51:02.188156 25111 heartbeater.cc:464] Registering TS with master...
I20260926 20:51:02.189024 25111 heartbeater.cc:511] Master 127.5.78.62:39327 requested a full tablet report, sending...
I20260926 20:51:02.190760 24788 ts_manager.cc:194] Registered new tserver with Master: b5617dd418834d66bbae7ddf37b71421 (127.5.78.2:44553)
I20260926 20:51:02.192013 24788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.2@KRBTEST.COM'} at 127.5.78.2:52817
WARNING: no policy specified for kudu/127.5.78.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
Entry for principal kudu/127.5.78.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab.
I20260926 20:51:02.226186 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39327
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:02.579202 25117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:02.579579 25117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:02.579763 25117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:02.587792 25117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:02.587966 25117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:02.588150 25117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:51:02.598619 25117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.3.keytab
--principal=kudu/127.5.78.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:39327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:02.600000 25117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:02.601364 25117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:02.615862 25122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:02.615769 25125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:02.615819 25123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:02.616765 25117 server_base.cc:1034] running on GCE node
I20260926 20:51:02.617437 25117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:02.618570 25117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:02.619820 25117 hybrid_clock.cc:648] HybridClock initialized: now 1790455862619763 us; error 43 us; skew 500 ppm
I20260926 20:51:02.620270 25117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:02 dist-test-slave-z7j8 krb5kdc[24743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455862, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:02.623816 25117 init.cc:377] Logged in from keytab as kudu/127.5.78.3@KRBTEST.COM (short username kudu)
I20260926 20:51:02.626610 25117 webserver.cc:460] Webserver started at http://127.5.78.3:43365/ using document root <none> and password file <none>
I20260926 20:51:02.627391 25117 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:02.627609 25117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:02.627976 25117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:02.631440 25117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "550470fa84a64730a9e5e35184365235"
format_stamp: "Formatted at 2026-09-26 20:51:02 on dist-test-slave-z7j8"
server_key: "fa815888347cc3417d882fbce58b470f"
server_key_iv: "d5e256f54ed40738f74be5382531c701"
server_key_version: "encryptionkey@0"
I20260926 20:51:02.632303 25117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "550470fa84a64730a9e5e35184365235"
format_stamp: "Formatted at 2026-09-26 20:51:02 on dist-test-slave-z7j8"
server_key: "fa815888347cc3417d882fbce58b470f"
server_key_iv: "d5e256f54ed40738f74be5382531c701"
server_key_version: "encryptionkey@0"
I20260926 20:51:02.638020 25117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:51:02.642552 25132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:02.644250 25117 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:51:02.644470 25117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "550470fa84a64730a9e5e35184365235"
format_stamp: "Formatted at 2026-09-26 20:51:02 on dist-test-slave-z7j8"
server_key: "fa815888347cc3417d882fbce58b470f"
server_key_iv: "d5e256f54ed40738f74be5382531c701"
server_key_version: "encryptionkey@0"
I20260926 20:51:02.644716 25117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:02.659677 24977 heartbeater.cc:503] Master 127.5.78.62:39327 was elected leader, sending a full tablet report...
I20260926 20:51:02.669474 25117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:02.674185 25117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:02.674577 25117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:02.675676 25117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:51:02.677171 25117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:51:02.677284 25117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:02.677441 25117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:51:02.677518 25117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:02.712672 25117 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:44345
I20260926 20:51:02.712707 25244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:44345 every 8 connection(s)
I20260926 20:51:02.714013 25117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:51:02.722888 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25117
I20260926 20:51:02.723091 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:51:02.723531 5432 external_mini_cluster.cc:1428] Setting key d0ab72a21e56e96b57a20596cfa16d25
Sep 26 20:51:02 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455862, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.3@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:02.737874 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.716690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:60835 (local address 127.5.78.62:39327)
0926 20:51:02.717153 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:02.717158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:02.718320 (+ 1162us) server_negotiation.cc:407] Connection header received
0926 20:51:02.720646 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:02.720652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:02.720766 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:02.720912 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:02.723030 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.724105 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.726453 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.727003 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.727155 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.731371 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:02.731377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:02.731380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:02.731427 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:02.734196 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:02.735235 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:02.735240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:02.735242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:02.735381 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:02.735870 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:02.735874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:02.735877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:02.736178 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:02.736342 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:02.737192 (+ 850us) server_negotiation.cc:299] Negotiation successful
0926 20:51:02.737485 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260926 20:51:02.739230 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.717305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:39327 (local address 127.5.78.3:60835)
0926 20:51:02.718126 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:02.718168 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:02.720164 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:02.721212 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:02.721223 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:02.721959 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:02.722765 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.722791 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.724369 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.724374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.726237 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.726253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.727264 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.727270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.727515 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.728354 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:02.728380 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:02.731106 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:02.734387 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:02.734396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:02.734490 (+ 94us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:02.735060 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:02.735509 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:02.735514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:02.735518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:02.735728 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:02.736315 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:02.736324 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:02.736700 (+ 376us) client_negotiation.cc:769] Sending connection context
0926 20:51:02.737000 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 20:51:02.737629 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":312,"threads_started":1}
I20260926 20:51:02.740868 25245 heartbeater.cc:347] Connected to a master server at 127.5.78.62:39327
I20260926 20:51:02.741215 25245 heartbeater.cc:464] Registering TS with master...
I20260926 20:51:02.742010 25245 heartbeater.cc:511] Master 127.5.78.62:39327 requested a full tablet report, sending...
I20260926 20:51:02.743618 24788 ts_manager.cc:194] Registered new tserver with Master: 550470fa84a64730a9e5e35184365235 (127.5.78.3:44345)
I20260926 20:51:02.744460 24788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.78.3@KRBTEST.COM'} at 127.5.78.3:60835
I20260926 20:51:02.752239 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:51:02.770362 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.756663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.5.78.62:39327)
0926 20:51:02.756832 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:02.756837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:02.757267 (+ 430us) server_negotiation.cc:407] Connection header received
0926 20:51:02.757405 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:02.757409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:02.757480 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:02.757575 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:02.759109 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.759870 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.761466 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.761916 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.762025 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.764498 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:02.764504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:02.764507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:02.764546 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:02.767574 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:02.768416 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:02.768421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:02.768423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:02.768562 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:02.768931 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:02.768935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:02.768937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:02.769306 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:02.769504 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:02.769874 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 20:51:02.770094 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:51:02.774935 24788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:51:02.781908 24788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:51:02.806155 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.796136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.5.78.2:44553)
0926 20:51:02.796800 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:02.796809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:02.797036 (+ 227us) server_negotiation.cc:407] Connection header received
0926 20:51:02.797343 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:02.797354 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:02.797610 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:02.797845 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:02.798982 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.800588 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.803516 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.804969 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.805110 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.805324 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:02.805625 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 20:51:02.805801 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":348,"threads_started":1}
I20260926 20:51:02.806263 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.796121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:44553 (local address 127.0.0.1:60652)
0926 20:51:02.796895 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:02.796927 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:02.797130 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:02.798212 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:02.798229 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:02.798273 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:02.798790 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.798807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.800735 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.800739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.803314 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.803327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.805166 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.805172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.805371 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.805422 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 20:51:02.805559 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 20:51:02.805774 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":66,"thread_start_us":339,"threads_started":1}
I20260926 20:51:02.807196 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.797233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:44345 (local address 127.0.0.1:36866)
0926 20:51:02.797796 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:02.797813 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:02.797922 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:02.799773 (+ 1851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:02.799778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:02.799803 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:02.800257 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.800269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.802296 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.802300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.804537 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.804546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.806229 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.806237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.806462 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.806543 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 20:51:02.806623 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 20:51:02.806766 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":30,"thread_start_us":278,"threads_started":1}
I20260926 20:51:02.807670 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.797864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:33639 (local address 127.0.0.1:59348)
0926 20:51:02.798463 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:02.798479 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:02.798596 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:02.800123 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:02.800128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:02.800151 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:02.800645 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.800661 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.801937 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.801945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.804315 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:02.804329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.806461 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.806467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:02.807046 (+ 579us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.807081 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:51:02.807169 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 20:51:02.807290 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":139,"thread_start_us":1181,"threads_started":1}
I20260926 20:51:02.808333 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.797493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.5.78.3:44345)
0926 20:51:02.799168 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:02.799176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:02.799205 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:51:02.799302 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:02.799310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:02.799564 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:02.800310 (+ 746us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:02.800559 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.802154 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.804680 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.805978 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.807555 (+ 1577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.807936 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:02.807994 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 20:51:02.808108 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1486,"thread_start_us":1343,"threads_started":1}
I20260926 20:51:02.808370 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:02.798470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.5.78.1:33639)
0926 20:51:02.799583 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:02.799591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:02.799617 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:51:02.799714 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:02.799721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:02.799910 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:02.800102 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:02.800827 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.801798 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.804476 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:02.805944 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:02.806067 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:02.806302 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:02.807897 (+ 1595us) server_negotiation.cc:299] Negotiation successful
0926 20:51:02.808062 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":1324,"threads_started":1}
I20260926 20:51:02.810302 25046 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac56a647fd6406f95786e2d10287960 (DEFAULT_TABLE table=test-table [id=39dbaef7100540e68d89c6e5f9c96a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:51:02.811527 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac56a647fd6406f95786e2d10287960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:02.813104 25180 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac56a647fd6406f95786e2d10287960 (DEFAULT_TABLE table=test-table [id=39dbaef7100540e68d89c6e5f9c96a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:51:02.813313 24912 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac56a647fd6406f95786e2d10287960 (DEFAULT_TABLE table=test-table [id=39dbaef7100540e68d89c6e5f9c96a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:51:02.814574 25180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac56a647fd6406f95786e2d10287960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:02.814574 24912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac56a647fd6406f95786e2d10287960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:02.823109 25264 tablet_bootstrap.cc:492] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Bootstrap starting.
I20260926 20:51:02.826785 25264 tablet_bootstrap.cc:654] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:02.827976 25264 log.cc:824] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:02.830616 25267 tablet_bootstrap.cc:492] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Bootstrap starting.
I20260926 20:51:02.830746 25266 tablet_bootstrap.cc:492] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Bootstrap starting.
I20260926 20:51:02.831401 25264 tablet_bootstrap.cc:492] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: No bootstrap required, opened a new log
I20260926 20:51:02.831691 25264 ts_tablet_manager.cc:1397] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:51:02.834335 25267 tablet_bootstrap.cc:654] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:02.834338 25266 tablet_bootstrap.cc:654] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:02.835711 25266 log.cc:824] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:02.835711 25267 log.cc:824] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:02.838639 25266 tablet_bootstrap.cc:492] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: No bootstrap required, opened a new log
I20260926 20:51:02.838917 25266 ts_tablet_manager.cc:1397] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260926 20:51:02.839032 25267 tablet_bootstrap.cc:492] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: No bootstrap required, opened a new log
I20260926 20:51:02.839311 25267 ts_tablet_manager.cc:1397] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260926 20:51:02.840847 25264 raft_consensus.cc:348] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:02.841291 25264 raft_consensus.cc:374] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:02.841434 25264 raft_consensus.cc:729] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5617dd418834d66bbae7ddf37b71421, State: Initialized, Role: FOLLOWER
I20260926 20:51:02.842083 25264 consensus_queue.cc:260] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:02.843233 25111 heartbeater.cc:503] Master 127.5.78.62:39327 was elected leader, sending a full tablet report...
I20260926 20:51:02.843885 25264 ts_tablet_manager.cc:1428] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260926 20:51:02.847718 25266 raft_consensus.cc:348] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:02.848201 25266 raft_consensus.cc:374] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:02.848382 25266 raft_consensus.cc:729] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550470fa84a64730a9e5e35184365235, State: Initialized, Role: FOLLOWER
I20260926 20:51:02.849241 25266 consensus_queue.cc:260] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:02.849694 25267 raft_consensus.cc:348] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:02.850236 25267 raft_consensus.cc:374] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:02.850382 25267 raft_consensus.cc:729] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db451fd1568345d1a9019234ea606fef, State: Initialized, Role: FOLLOWER
I20260926 20:51:02.850852 25245 heartbeater.cc:503] Master 127.5.78.62:39327 was elected leader, sending a full tablet report...
I20260926 20:51:02.851086 25267 consensus_queue.cc:260] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:02.851295 25266 ts_tablet_manager.cc:1428] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260926 20:51:02.852873 25267 ts_tablet_manager.cc:1428] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Time spent starting tablet: real 0.013s user 0.007s sys 0.007s
W20260926 20:51:02.874493 24978 tablet.cc:2369] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:51:02.914317 25112 tablet.cc:2369] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:51:02.965751 25246 tablet.cc:2369] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:51:03.031549 25270 raft_consensus.cc:482] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:51:03.031872 25270 raft_consensus.cc:504] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:03.034006 25270 leader_election.cc:288] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 550470fa84a64730a9e5e35184365235 (127.5.78.3:44345), db451fd1568345d1a9019234ea606fef (127.5.78.1:33639)
I20260926 20:51:03.040725 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.034531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:55149 (local address 127.5.78.3:44345)
0926 20:51:03.034775 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:03.034784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:03.035207 (+ 423us) server_negotiation.cc:407] Connection header received
0926 20:51:03.035476 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:03.035485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:03.035585 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:03.035686 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:03.036318 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.037071 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.038960 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.039737 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.039897 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.040010 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:03.040291 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 20:51:03.040426 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:51:03.040809 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.034387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:44345 (local address 127.5.78.2:55149)
0926 20:51:03.035089 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:03.035106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:03.035278 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:03.035762 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:03.035767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:03.035792 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:03.036147 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:03.036156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.037227 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.037232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:03.038788 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:03.038801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.039961 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.039967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:03.040119 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.040143 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 20:51:03.040264 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 20:51:03.040426 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":357,"threads_started":1}
I20260926 20:51:03.041074 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.035492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:54113 (local address 127.5.78.1:33639)
0926 20:51:03.035711 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:03.035717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:03.035948 (+ 231us) server_negotiation.cc:407] Connection header received
0926 20:51:03.036159 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:03.036164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:03.036238 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:03.036337 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:03.036870 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.037584 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.039319 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.040089 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.040271 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.040377 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:03.040686 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 20:51:03.040825 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:51:03.041139 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.035275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:33639 (local address 127.5.78.2:54113)
0926 20:51:03.035845 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:03.035862 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:03.036006 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:03.036351 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:03.036356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:03.036379 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:03.036720 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:03.036729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.037734 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.037740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:03.039151 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:03.039164 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.040335 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.040341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:03.040523 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.040546 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:51:03.040656 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 20:51:03.040805 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":259,"threads_started":1}
I20260926 20:51:03.042026 25200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac56a647fd6406f95786e2d10287960" candidate_uuid: "b5617dd418834d66bbae7ddf37b71421" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550470fa84a64730a9e5e35184365235" is_pre_election: true
I20260926 20:51:03.042138 24932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac56a647fd6406f95786e2d10287960" candidate_uuid: "b5617dd418834d66bbae7ddf37b71421" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db451fd1568345d1a9019234ea606fef" is_pre_election: true
I20260926 20:51:03.042606 25200 raft_consensus.cc:2454] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5617dd418834d66bbae7ddf37b71421 in term 0.
I20260926 20:51:03.042688 24932 raft_consensus.cc:2454] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5617dd418834d66bbae7ddf37b71421 in term 0.
I20260926 20:51:03.043457 25001 leader_election.cc:302] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550470fa84a64730a9e5e35184365235, b5617dd418834d66bbae7ddf37b71421; no voters:
I20260926 20:51:03.043905 25270 raft_consensus.cc:2781] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:51:03.044081 25270 raft_consensus.cc:482] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:51:03.044245 25270 raft_consensus.cc:3037] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:03.046589 25270 raft_consensus.cc:504] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:03.047251 25270 leader_election.cc:288] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [CANDIDATE]: Term 1 election: Requested vote from peers 550470fa84a64730a9e5e35184365235 (127.5.78.3:44345), db451fd1568345d1a9019234ea606fef (127.5.78.1:33639)
I20260926 20:51:03.047933 25200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac56a647fd6406f95786e2d10287960" candidate_uuid: "b5617dd418834d66bbae7ddf37b71421" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "550470fa84a64730a9e5e35184365235"
I20260926 20:51:03.048017 24932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac56a647fd6406f95786e2d10287960" candidate_uuid: "b5617dd418834d66bbae7ddf37b71421" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db451fd1568345d1a9019234ea606fef"
I20260926 20:51:03.048177 25200 raft_consensus.cc:3037] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:03.048353 24932 raft_consensus.cc:3037] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:03.050761 25200 raft_consensus.cc:2454] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5617dd418834d66bbae7ddf37b71421 in term 1.
I20260926 20:51:03.051039 24932 raft_consensus.cc:2454] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5617dd418834d66bbae7ddf37b71421 in term 1.
I20260926 20:51:03.051367 25001 leader_election.cc:302] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550470fa84a64730a9e5e35184365235, b5617dd418834d66bbae7ddf37b71421; no voters:
I20260926 20:51:03.051740 25270 raft_consensus.cc:2781] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:03.052229 25270 raft_consensus.cc:686] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 1 LEADER]: Becoming Leader. State: Replica: b5617dd418834d66bbae7ddf37b71421, State: Running, Role: LEADER
I20260926 20:51:03.052728 25270 consensus_queue.cc:237] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } }
I20260926 20:51:03.057976 24787 catalog_manager.cc:5495] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5617dd418834d66bbae7ddf37b71421 (127.5.78.2). New cstate: current_term: 1 leader_uuid: "b5617dd418834d66bbae7ddf37b71421" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5617dd418834d66bbae7ddf37b71421" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 44553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 } health_report { overall_health: UNKNOWN } } }
I20260926 20:51:03.103894 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.098445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.5.78.2:44553)
0926 20:51:03.098658 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:03.098663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:03.098682 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:51:03.098843 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:03.098847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:03.098927 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:03.099059 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:03.099420 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.100158 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.101625 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.102018 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.102114 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.102689 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:03.103203 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:03.103326 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:03.103475 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 20:51:03.103548 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:51:03.112921 24932 raft_consensus.cc:1261] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 1 FOLLOWER]: Refusing update from remote peer b5617dd418834d66bbae7ddf37b71421: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:03.112921 25200 raft_consensus.cc:1261] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 1 FOLLOWER]: Refusing update from remote peer b5617dd418834d66bbae7ddf37b71421: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:03.114182 25275 consensus_queue.cc:1035] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [LEADER]: Connected to new peer: Peer: permanent_uuid: "550470fa84a64730a9e5e35184365235" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:03.114706 25270 consensus_queue.cc:1035] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db451fd1568345d1a9019234ea606fef" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 33639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:03.144569 24787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:51:03.144893 24787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:51:03.149293 24787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f521cab210eb4599a61fb2d5c23f6cf4: Sending DeleteTablet for 3 replicas of tablet 6ac56a647fd6406f95786e2d10287960
I20260926 20:51:03.150772 25046 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac56a647fd6406f95786e2d10287960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:51:03 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:60652
I20260926 20:51:03.150972 24912 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac56a647fd6406f95786e2d10287960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:51:03 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:59348
I20260926 20:51:03.152067 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24849
I20260926 20:51:03.151978 25180 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac56a647fd6406f95786e2d10287960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:51:03 UTC) from {username='kudu', principal='kudu/127.5.78.62@KRBTEST.COM'} at 127.0.0.1:36866
I20260926 20:51:03.165653 25290 tablet_replica.cc:330] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: stopping tablet replica
I20260926 20:51:03.168067 25290 raft_consensus.cc:2229] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:51:03.169050 25289 tablet_replica.cc:330] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: stopping tablet replica
I20260926 20:51:03.169714 25289 raft_consensus.cc:2229] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:51:03.170122 25290 raft_consensus.cc:2258] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:51:03.170303 25289 raft_consensus.cc:2258] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:51:03.171231 25288 tablet_replica.cc:330] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: stopping tablet replica
I20260926 20:51:03.173171 25289 ts_tablet_manager.cc:1905] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:51:03.173171 25290 ts_tablet_manager.cc:1905] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:51:03.173734 25288 raft_consensus.cc:2229] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:51:03.175143 25288 raft_consensus.cc:2258] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 20:51:03 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.1@KRBTEST.COM
I20260926 20:51:03.180915 25290 ts_tablet_manager.cc:1918] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:51:03.181236 25290 log.cc:1196] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/6ac56a647fd6406f95786e2d10287960
I20260926 20:51:03.181855 25290 ts_tablet_manager.cc:1939] T 6ac56a647fd6406f95786e2d10287960 P 550470fa84a64730a9e5e35184365235: Deleting consensus metadata
I20260926 20:51:03.181887 25288 ts_tablet_manager.cc:1905] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:51:03.184712 24774 catalog_manager.cc:4841] TS 550470fa84a64730a9e5e35184365235 (127.5.78.3:44345): tablet 6ac56a647fd6406f95786e2d10287960 (table test-table [id=39dbaef7100540e68d89c6e5f9c96a35]) successfully deleted
I20260926 20:51:03.188282 25289 ts_tablet_manager.cc:1918] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:51:03.188648 25289 log.cc:1196] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/6ac56a647fd6406f95786e2d10287960
I20260926 20:51:03.189417 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.153607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.5.78.1:33639)
0926 20:51:03.153841 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:03.153849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:03.164606 (+ 10757us) server_negotiation.cc:407] Connection header received
0926 20:51:03.165956 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:03.165963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:03.166085 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:03.166193 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:03.171018 (+ 4825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.172247 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.174304 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.174968 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.175076 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.181094 (+ 6018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:03.181112 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:03.181134 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:03.181185 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:03.185255 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:03.186301 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:03.186312 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:03.186329 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:03.186567 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:03.187106 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:03.187113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:03.187119 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:03.187624 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:03.187946 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:03.188656 (+ 710us) server_negotiation.cc:299] Negotiation successful
0926 20:51:03.188966 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:51:03.189828 25288 ts_tablet_manager.cc:1918] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:51:03.189391 25289 ts_tablet_manager.cc:1939] T 6ac56a647fd6406f95786e2d10287960 P db451fd1568345d1a9019234ea606fef: Deleting consensus metadata
I20260926 20:51:03.190111 25288 log.cc:1196] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/6ac56a647fd6406f95786e2d10287960
I20260926 20:51:03.190788 25288 ts_tablet_manager.cc:1939] T 6ac56a647fd6406f95786e2d10287960 P b5617dd418834d66bbae7ddf37b71421: Deleting consensus metadata
I20260926 20:51:03.190912 24972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:03.192629 24774 catalog_manager.cc:4841] TS b5617dd418834d66bbae7ddf37b71421 (127.5.78.2:44553): tablet 6ac56a647fd6406f95786e2d10287960 (table test-table [id=39dbaef7100540e68d89c6e5f9c96a35]) successfully deleted
I20260926 20:51:03.313306 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24849
I20260926 20:51:03.313417 24774 catalog_manager.cc:4841] TS db451fd1568345d1a9019234ea606fef (127.5.78.1:33639): tablet 6ac56a647fd6406f95786e2d10287960 (table test-table [id=39dbaef7100540e68d89c6e5f9c96a35]) successfully deleted
I20260926 20:51:03.332636 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24983
Sep 26 20:51:03 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.2@KRBTEST.COM
I20260926 20:51:03.352677 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.333723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.5.78.2:44553)
0926 20:51:03.333966 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:03.333974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:03.334036 (+ 62us) server_negotiation.cc:407] Connection header received
0926 20:51:03.334113 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:03.334124 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:03.334215 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:03.334342 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:03.335892 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.337210 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.338737 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.339503 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.339668 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.345156 (+ 5488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:03.345178 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:03.345195 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:03.345262 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:03.349100 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:03.350208 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:03.350217 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:03.350219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:03.350400 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:03.350975 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:03.350992 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:03.350995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:03.351414 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:03.351766 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:03.352123 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 20:51:03.352322 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 20:51:03.353468 25106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:03.469813 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24983
I20260926 20:51:03.488659 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25117
Sep 26 20:51:03 dist-test-slave-z7j8 krb5kdc[24743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.3@KRBTEST.COM
I20260926 20:51:03.518115 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:03.497577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48162 (local address 127.5.78.3:44345)
0926 20:51:03.497836 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:03.497841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:03.497867 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:51:03.497953 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:03.497961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:03.498071 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:03.498224 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:03.500065 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.501480 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.503154 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:03.504007 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:03.504174 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:03.509712 (+ 5538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:03.509737 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:03.509763 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:03.509824 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:03.514584 (+ 4760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:03.515462 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:03.515476 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:03.515482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:03.515672 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:03.516118 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:03.516125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:03.516130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:03.516650 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:03.516997 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:03.517264 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 20:51:03.517563 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 20:51:03.518877 25240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:03.609079 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25117
I20260926 20:51:03.626343 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 24756
I20260926 20:51:03.627602 24818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:03.738968 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 24756
2026-09-26T20:51:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4595 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T20:51:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:03Z Disabled control of system clock
I20260926 20:51:03.816847 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:35697
--webserver_interface=127.5.78.62
--webserver_port=0
--builtin_ntp_servers=127.5.78.20:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:35697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:51:04.162204 25299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:04.162614 25299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:04.162706 25299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:04.170334 25299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:04.170518 25299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:04.170562 25299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:04.170632 25299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:04.170686 25299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:04.180845 25299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:35697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:35697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:04.182305 25299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:04.184264 25299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:04.196558 25304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:04.196698 25305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:04.196702 25307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:04.197463 25299 server_base.cc:1034] running on GCE node
I20260926 20:51:04.198320 25299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:04.199862 25299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:04.201104 25299 hybrid_clock.cc:648] HybridClock initialized: now 1790455864201052 us; error 46 us; skew 500 ppm
I20260926 20:51:04.201504 25299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:51:04.203936 25299 webserver.cc:460] Webserver started at http://127.5.78.62:45391/ using document root <none> and password file <none>
I20260926 20:51:04.204641 25299 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:04.204771 25299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:04.205088 25299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:04.208118 25299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "cf1b9999dc62424082fc96d79201af13"
format_stamp: "Formatted at 2026-09-26 20:51:04 on dist-test-slave-z7j8"
server_key: "e6b8cfe705fae03947c5087d68c31c0d"
server_key_iv: "f2c6ec7192bbbc4a9bf34ec53773da53"
server_key_version: "encryptionkey@0"
I20260926 20:51:04.208947 25299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "cf1b9999dc62424082fc96d79201af13"
format_stamp: "Formatted at 2026-09-26 20:51:04 on dist-test-slave-z7j8"
server_key: "e6b8cfe705fae03947c5087d68c31c0d"
server_key_iv: "f2c6ec7192bbbc4a9bf34ec53773da53"
server_key_version: "encryptionkey@0"
I20260926 20:51:04.215397 25299 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 20:51:04.219183 25313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:04.220644 25299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:04.220860 25299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "cf1b9999dc62424082fc96d79201af13"
format_stamp: "Formatted at 2026-09-26 20:51:04 on dist-test-slave-z7j8"
server_key: "e6b8cfe705fae03947c5087d68c31c0d"
server_key_iv: "f2c6ec7192bbbc4a9bf34ec53773da53"
server_key_version: "encryptionkey@0"
I20260926 20:51:04.221101 25299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:04.239662 25299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:04.240701 25299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:04.241045 25299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:04.262084 25299 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:35697
I20260926 20:51:04.262104 25364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:35697 every 8 connection(s)
I20260926 20:51:04.263844 25299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:04.267974 25365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:04.272188 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25299
I20260926 20:51:04.272399 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:04.272828 5432 external_mini_cluster.cc:1428] Setting key cc92e5cd2fd0ca136def225742e93627
I20260926 20:51:04.280525 25365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13: Bootstrap starting.
I20260926 20:51:04.284026 25365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:04.285269 25365 log.cc:824] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:04.288475 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:04.274830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.5.78.62:35697)
0926 20:51:04.275951 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:04.275966 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:04.276013 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:51:04.278919 (+ 2906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:04.278952 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:04.279513 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:04.279965 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:04.280607 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:04.282004 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:04.283530 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:04.284320 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:04.284543 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:04.285804 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:04.285818 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:04.285846 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:04.286688 (+ 842us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:04.286802 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:04.286826 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:04.286956 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:04.287197 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 20:51:04.287530 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":395,"threads_started":1}
I20260926 20:51:04.288517 25365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13: No bootstrap required, opened a new log
I20260926 20:51:04.297535 25365 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1b9999dc62424082fc96d79201af13" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35697 } }
I20260926 20:51:04.297979 25365 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:04.298095 25365 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1b9999dc62424082fc96d79201af13, State: Initialized, Role: FOLLOWER
I20260926 20:51:04.298821 25365 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1b9999dc62424082fc96d79201af13" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35697 } }
I20260926 20:51:04.299036 25365 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:04.299199 25365 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:04.299376 25365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:04.301692 25365 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1b9999dc62424082fc96d79201af13" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35697 } }
I20260926 20:51:04.302284 25365 leader_election.cc:302] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf1b9999dc62424082fc96d79201af13; no voters:
I20260926 20:51:04.303015 25365 leader_election.cc:288] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:04.303174 25370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:04.304625 25370 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [term 1 LEADER]: Becoming Leader. State: Replica: cf1b9999dc62424082fc96d79201af13, State: Running, Role: LEADER
I20260926 20:51:04.305178 25370 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1b9999dc62424082fc96d79201af13" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35697 } }
I20260926 20:51:04.305493 25365 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:04.310869 25372 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1b9999dc62424082fc96d79201af13. Latest consensus state: current_term: 1 leader_uuid: "cf1b9999dc62424082fc96d79201af13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1b9999dc62424082fc96d79201af13" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35697 } } }
I20260926 20:51:04.311357 25372 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:04.311848 25371 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf1b9999dc62424082fc96d79201af13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1b9999dc62424082fc96d79201af13" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 35697 } } }
I20260926 20:51:04.312184 25371 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:04.313184 25384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:04.318850 25384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:04.328634 25384 catalog_manager.cc:1347] Generated new cluster ID: 96d0192ef49b46a4b7072ad51a2a2b93
I20260926 20:51:04.328771 25384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:04.353910 25384 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:04.355446 25384 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:04.366319 25384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13: Generated new TSK 0
I20260926 20:51:04.367327 25384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:04.378397 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.1:0
--local_ip_for_outbound_sockets=127.5.78.1
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:35697
--builtin_ntp_servers=127.5.78.20:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:51:04.739868 25389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:04.740164 25389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:04.740310 25389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:04.747841 25389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:04.747995 25389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:04.748168 25389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.1
I20260926 20:51:04.757869 25389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.78.1
--webserver_port=0
--tserver_master_addrs=127.5.78.62:35697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.1
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:04.759351 25389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:04.761165 25389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:04.775028 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:04.774866 25397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:04.774775 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:04.776364 25389 server_base.cc:1034] running on GCE node
I20260926 20:51:04.777127 25389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:04.778157 25389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:04.779425 25389 hybrid_clock.cc:648] HybridClock initialized: now 1790455864779343 us; error 94 us; skew 500 ppm
I20260926 20:51:04.779879 25389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:51:04.782922 25389 webserver.cc:460] Webserver started at http://127.5.78.1:41829/ using document root <none> and password file <none>
I20260926 20:51:04.783941 25389 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:04.784065 25389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:04.784469 25389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:04.788120 25389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data/instance:
uuid: "cd4939bad2994cabac29092a21ed84c8"
format_stamp: "Formatted at 2026-09-26 20:51:04 on dist-test-slave-z7j8"
server_key: "1c692824fd99a389bf814368817119d7"
server_key_iv: "b427b895a8c2e3c0acbe922eead8f12e"
server_key_version: "encryptionkey@0"
I20260926 20:51:04.788893 25389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance:
uuid: "cd4939bad2994cabac29092a21ed84c8"
format_stamp: "Formatted at 2026-09-26 20:51:04 on dist-test-slave-z7j8"
server_key: "1c692824fd99a389bf814368817119d7"
server_key_iv: "b427b895a8c2e3c0acbe922eead8f12e"
server_key_version: "encryptionkey@0"
I20260926 20:51:04.794178 25389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:51:04.797650 25403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:04.799158 25389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:04.799328 25389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal
uuid: "cd4939bad2994cabac29092a21ed84c8"
format_stamp: "Formatted at 2026-09-26 20:51:04 on dist-test-slave-z7j8"
server_key: "1c692824fd99a389bf814368817119d7"
server_key_iv: "b427b895a8c2e3c0acbe922eead8f12e"
server_key_version: "encryptionkey@0"
I20260926 20:51:04.799551 25389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:04.815723 25389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:04.816680 25389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:04.816967 25389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:04.817971 25389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:51:04.819479 25389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:51:04.819571 25389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:04.819725 25389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:51:04.819828 25389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:04.853600 25389 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.1:46493
I20260926 20:51:04.853653 25515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.1:46493 every 8 connection(s)
I20260926 20:51:04.854949 25389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/data/info.pb
I20260926 20:51:04.860760 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25389
I20260926 20:51:04.860944 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal/instance
I20260926 20:51:04.861521 5432 external_mini_cluster.cc:1428] Setting key 3643020ed7b389a395ab6942ab5b33fd
I20260926 20:51:04.864506 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.2:0
--local_ip_for_outbound_sockets=127.5.78.2
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:35697
--builtin_ntp_servers=127.5.78.20:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:51:04.869027 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:04.857632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.1:38727 (local address 127.5.78.62:35697)
0926 20:51:04.858150 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:04.858160 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:04.859235 (+ 1075us) server_negotiation.cc:407] Connection header received
0926 20:51:04.861162 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:04.861168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:04.861262 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:04.861389 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:04.862647 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:04.863625 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:04.865371 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:04.866216 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:04.866359 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:04.867933 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:04.867942 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:04.867950 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:04.867978 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:04.868032 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:04.868049 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:04.868147 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:04.868443 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 20:51:04.868563 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":263,"threads_started":1}
I20260926 20:51:04.870040 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:04.858142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:35697 (local address 127.5.78.1:38727)
0926 20:51:04.859060 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:04.859099 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:04.860816 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:04.861582 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:04.861590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:04.861743 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:04.862413 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:04.862473 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:04.863806 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:04.863810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:04.865208 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:04.865221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:04.866484 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:04.866493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:04.866787 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:04.867550 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:51:04.867553 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:04.867597 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:51:04.867708 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:04.868202 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:04.868212 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:04.868220 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 20:51:04.868384 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 20:51:04.868881 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":300,"threads_started":1}
I20260926 20:51:04.871948 25516 heartbeater.cc:347] Connected to a master server at 127.5.78.62:35697
I20260926 20:51:04.872313 25516 heartbeater.cc:464] Registering TS with master...
I20260926 20:51:04.873165 25516 heartbeater.cc:511] Master 127.5.78.62:35697 requested a full tablet report, sending...
I20260926 20:51:04.875694 25330 ts_manager.cc:194] Registered new tserver with Master: cd4939bad2994cabac29092a21ed84c8 (127.5.78.1:46493)
I20260926 20:51:04.877985 25330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.1:38727
W20260926 20:51:05.211555 25520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:05.211858 25520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:05.211943 25520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:05.220791 25520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:05.220904 25520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:05.221014 25520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.2
I20260926 20:51:05.230701 25520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.78.2
--webserver_port=0
--tserver_master_addrs=127.5.78.62:35697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.2
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:05.231950 25520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:05.233664 25520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:05.247709 25525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:05.247748 25526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:05.248037 25528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:05.248684 25520 server_base.cc:1034] running on GCE node
I20260926 20:51:05.249321 25520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:05.250367 25520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:05.251680 25520 hybrid_clock.cc:648] HybridClock initialized: now 1790455865251594 us; error 79 us; skew 500 ppm
I20260926 20:51:05.252036 25520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:51:05.254693 25520 webserver.cc:460] Webserver started at http://127.5.78.2:39383/ using document root <none> and password file <none>
I20260926 20:51:05.255472 25520 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:05.255564 25520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:05.255946 25520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:05.258986 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data/instance:
uuid: "efde0739814b4d90b967971ca38234ff"
format_stamp: "Formatted at 2026-09-26 20:51:05 on dist-test-slave-z7j8"
server_key: "a2c77e5049fa8c924282f6f8c23620a1"
server_key_iv: "abd020b064ce322a1abf05501f0da515"
server_key_version: "encryptionkey@0"
I20260926 20:51:05.259814 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance:
uuid: "efde0739814b4d90b967971ca38234ff"
format_stamp: "Formatted at 2026-09-26 20:51:05 on dist-test-slave-z7j8"
server_key: "a2c77e5049fa8c924282f6f8c23620a1"
server_key_iv: "abd020b064ce322a1abf05501f0da515"
server_key_version: "encryptionkey@0"
I20260926 20:51:05.266999 25520 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 20:51:05.271906 25534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:05.273722 25520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:51:05.273898 25520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal
uuid: "efde0739814b4d90b967971ca38234ff"
format_stamp: "Formatted at 2026-09-26 20:51:05 on dist-test-slave-z7j8"
server_key: "a2c77e5049fa8c924282f6f8c23620a1"
server_key_iv: "abd020b064ce322a1abf05501f0da515"
server_key_version: "encryptionkey@0"
I20260926 20:51:05.274145 25520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:05.300313 25520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:05.301170 25520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:05.301416 25520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:05.302397 25520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:51:05.304100 25520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:51:05.304189 25520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:05.304306 25520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:51:05.304379 25520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:05.338621 25520 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.2:38583
I20260926 20:51:05.338726 25646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.2:38583 every 8 connection(s)
I20260926 20:51:05.340003 25520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/data/info.pb
I20260926 20:51:05.346019 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25520
I20260926 20:51:05.346218 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal/instance
I20260926 20:51:05.346797 5432 external_mini_cluster.cc:1428] Setting key 88ed547a63d0a6b868a8dcd2e81c0a8b
I20260926 20:51:05.350132 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.78.3:0
--local_ip_for_outbound_sockets=127.5.78.3
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:35697
--builtin_ntp_servers=127.5.78.20:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:51:05.356025 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.342941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.2:33413 (local address 127.5.78.62:35697)
0926 20:51:05.343152 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:05.343158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:05.344683 (+ 1525us) server_negotiation.cc:407] Connection header received
0926 20:51:05.346726 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:05.346734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:05.346850 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:05.346980 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:05.348408 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.349377 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.352215 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.352638 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.352745 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.354970 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:05.354977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:05.354982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:05.355008 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:05.355050 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:05.355069 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:05.355181 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:05.355486 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 20:51:05.355632 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:51:05.357434 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.343496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:35697 (local address 127.5.78.2:33413)
0926 20:51:05.344491 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:05.344535 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:05.346337 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:05.347270 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:05.347280 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:05.347418 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:05.348203 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.348225 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.349575 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.349582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.352028 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.352048 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.352906 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.352916 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.353268 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.354512 (+ 1244us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:51:05.354518 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:05.354577 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:51:05.354698 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:05.355194 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:05.355208 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:05.355219 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:51:05.355417 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 20:51:05.355925 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":299,"threads_started":1}
I20260926 20:51:05.359789 25647 heartbeater.cc:347] Connected to a master server at 127.5.78.62:35697
I20260926 20:51:05.360175 25647 heartbeater.cc:464] Registering TS with master...
I20260926 20:51:05.361222 25647 heartbeater.cc:511] Master 127.5.78.62:35697 requested a full tablet report, sending...
I20260926 20:51:05.363358 25330 ts_manager.cc:194] Registered new tserver with Master: efde0739814b4d90b967971ca38234ff (127.5.78.2:38583)
I20260926 20:51:05.364491 25330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.2:33413
W20260926 20:51:05.706743 25650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:05.707111 25650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:05.707293 25650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:05.715304 25650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:05.715485 25650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:05.715695 25650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.78.3
I20260926 20:51:05.725595 25650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.78.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.78.3
--webserver_port=0
--tserver_master_addrs=127.5.78.62:35697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.78.3
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:05.727087 25650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:05.729050 25650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:05.744349 25655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:05.744289 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:05.744267 25658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:05.745038 25650 server_base.cc:1034] running on GCE node
I20260926 20:51:05.745893 25650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:05.747128 25650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:05.748392 25650 hybrid_clock.cc:648] HybridClock initialized: now 1790455865748323 us; error 50 us; skew 500 ppm
I20260926 20:51:05.748732 25650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:51:05.751339 25650 webserver.cc:460] Webserver started at http://127.5.78.3:36771/ using document root <none> and password file <none>
I20260926 20:51:05.752094 25650 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:05.752193 25650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:05.752491 25650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:05.755642 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data/instance:
uuid: "69c4718482cc4de282846fc6240ae5d8"
format_stamp: "Formatted at 2026-09-26 20:51:05 on dist-test-slave-z7j8"
server_key: "4cf2c94ef967312f00769951e1314c9f"
server_key_iv: "e1611f9ef814e45b5224de1b19e761cf"
server_key_version: "encryptionkey@0"
I20260926 20:51:05.756481 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance:
uuid: "69c4718482cc4de282846fc6240ae5d8"
format_stamp: "Formatted at 2026-09-26 20:51:05 on dist-test-slave-z7j8"
server_key: "4cf2c94ef967312f00769951e1314c9f"
server_key_iv: "e1611f9ef814e45b5224de1b19e761cf"
server_key_version: "encryptionkey@0"
I20260926 20:51:05.762188 25650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:51:05.766731 25664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:05.768430 25650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:51:05.768626 25650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal
uuid: "69c4718482cc4de282846fc6240ae5d8"
format_stamp: "Formatted at 2026-09-26 20:51:05 on dist-test-slave-z7j8"
server_key: "4cf2c94ef967312f00769951e1314c9f"
server_key_iv: "e1611f9ef814e45b5224de1b19e761cf"
server_key_version: "encryptionkey@0"
I20260926 20:51:05.768844 25650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:05.809437 25650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:05.810511 25650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:05.810856 25650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:05.812011 25650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:51:05.813802 25650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:51:05.813901 25650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:05.814080 25650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:51:05.814194 25650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:05.850996 25650 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.3:44717
I20260926 20:51:05.851051 25776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.3:44717 every 8 connection(s)
I20260926 20:51:05.852460 25650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/data/info.pb
I20260926 20:51:05.853207 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25650
I20260926 20:51:05.853447 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal/instance
I20260926 20:51:05.854194 5432 external_mini_cluster.cc:1428] Setting key 66d8e364d34d1b052a5cb37bcb1b66b5
I20260926 20:51:05.872615 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.857152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:45711 (local address 127.5.78.62:35697)
0926 20:51:05.862880 (+ 5728us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:05.862885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:05.863060 (+ 175us) server_negotiation.cc:407] Connection header received
0926 20:51:05.864994 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:05.864999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:05.865082 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:05.865191 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:05.866614 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.867386 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.869155 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.869771 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.869907 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.871762 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:05.871767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:05.871770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:05.871790 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:05.871831 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:05.871844 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:05.871949 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:05.872245 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 20:51:05.872335 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5604,"thread_start_us":5849,"threads_started":1}
I20260926 20:51:05.874019 25779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.856563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:35697 (local address 127.5.78.3:45711)
0926 20:51:05.862849 (+ 6286us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:05.862901 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:05.864670 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:05.865496 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:05.865506 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:05.865627 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:05.866334 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.866354 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.867556 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.867559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.868968 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.868982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.870063 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.870073 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.870385 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.871373 (+ 988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:51:05.871377 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:05.871437 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:51:05.871550 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:05.871971 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:05.871981 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:05.871988 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:51:05.872171 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 20:51:05.872547 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5611,"thread_start_us":5367,"threads_started":1}
I20260926 20:51:05.875609 25777 heartbeater.cc:347] Connected to a master server at 127.5.78.62:35697
I20260926 20:51:05.875978 25777 heartbeater.cc:464] Registering TS with master...
I20260926 20:51:05.876802 25777 heartbeater.cc:511] Master 127.5.78.62:35697 requested a full tablet report, sending...
I20260926 20:51:05.878259 25330 ts_manager.cc:194] Registered new tserver with Master: 69c4718482cc4de282846fc6240ae5d8 (127.5.78.3:44717)
I20260926 20:51:05.879323 25330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.78.3:45711
I20260926 20:51:05.881278 25516 heartbeater.cc:503] Master 127.5.78.62:35697 was elected leader, sending a full tablet report...
I20260926 20:51:05.884008 5432 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:51:05.894529 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.888274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.5.78.62:35697)
0926 20:51:05.888482 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:05.888491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:05.888875 (+ 384us) server_negotiation.cc:407] Connection header received
0926 20:51:05.889075 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:05.889080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:05.889167 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:05.889271 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:05.889771 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.890582 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.891990 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.892405 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.892549 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.893671 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:05.893677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:05.893682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:05.893711 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:05.893746 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:05.893758 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:05.893897 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:05.894047 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 20:51:05.894146 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:51:05.898600 25330 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:51:05.906500 25330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:51:05.939376 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.920159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33356 (local address 127.5.78.2:38583)
0926 20:51:05.930755 (+ 10596us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:05.930764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:05.930792 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:51:05.930892 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:05.930900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:05.931162 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:05.931385 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:05.932351 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.933973 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.936489 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.938000 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.938144 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.938499 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:05.938794 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 20:51:05.939010 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10408,"thread_start_us":10474,"threads_started":1}
I20260926 20:51:05.939381 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.919982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:38583 (local address 127.0.0.1:33356)
0926 20:51:05.921188 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:05.921255 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:05.921486 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:05.931691 (+ 10205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:05.931698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:05.931728 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:05.932137 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.932151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.934142 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.934147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.936280 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.936292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.938234 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.938240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.938500 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.938570 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 20:51:05.938726 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 20:51:05.939002 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":30,"thread_start_us":9294,"threads_started":1}
I20260926 20:51:05.939476 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.920228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.3:44717 (local address 127.0.0.1:51640)
0926 20:51:05.929540 (+ 9312us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:05.929562 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:05.929743 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:05.930760 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:05.930768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:05.930807 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:05.931376 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.931392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.933525 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.933530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.935540 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.935554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.938657 (+ 3103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.938662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.938871 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.938905 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:51:05.938993 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 20:51:05.939124 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9182,"mutex_wait_us":1,"thread_start_us":846,"threads_started":1}
I20260926 20:51:05.940507 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.920715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.5.78.3:44717)
0926 20:51:05.929235 (+ 8520us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:05.929247 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:05.929960 (+ 713us) server_negotiation.cc:407] Connection header received
0926 20:51:05.930078 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:05.930086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:05.930467 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:05.931590 (+ 1123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:05.931885 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.933363 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.935742 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.938166 (+ 2424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.938289 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.938553 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:05.939942 (+ 1389us) server_negotiation.cc:299] Negotiation successful
0926 20:51:05.940121 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8316,"thread_start_us":8154,"threads_started":1}
I20260926 20:51:05.940671 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.924515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55186)
0926 20:51:05.931479 (+ 6964us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:05.931506 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:05.931676 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:05.933165 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:05.933171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:05.933199 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:05.933621 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.933634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.935933 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.935939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.937956 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:05.937972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.939951 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.939957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:05.940121 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.940162 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 20:51:05.940275 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 20:51:05.940402 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6785,"thread_start_us":6575,"threads_started":1}
I20260926 20:51:05.940898 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:05.925345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.5.78.1:46493)
0926 20:51:05.932238 (+ 6893us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:05.932245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:05.932297 (+ 52us) server_negotiation.cc:407] Connection header received
0926 20:51:05.932416 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:05.932425 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:05.932714 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:05.933837 (+ 1123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:05.934105 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.935776 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.938147 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:05.939720 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:05.939910 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:05.940184 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:05.940320 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 20:51:05.940494 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6722,"thread_start_us":6583,"threads_started":1}
I20260926 20:51:05.943969 25712 tablet_service.cc:1467] Processing CreateTablet for tablet 39d0a863b369423f89c7f4917bedeac8 (DEFAULT_TABLE table=test-table [id=739b4ec9e8284260bf4b5947041adb3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:51:05.944075 25451 tablet_service.cc:1467] Processing CreateTablet for tablet 39d0a863b369423f89c7f4917bedeac8 (DEFAULT_TABLE table=test-table [id=739b4ec9e8284260bf4b5947041adb3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:51:05.944075 25582 tablet_service.cc:1467] Processing CreateTablet for tablet 39d0a863b369423f89c7f4917bedeac8 (DEFAULT_TABLE table=test-table [id=739b4ec9e8284260bf4b5947041adb3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:51:05.945269 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39d0a863b369423f89c7f4917bedeac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:05.945420 25451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39d0a863b369423f89c7f4917bedeac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:05.945433 25582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39d0a863b369423f89c7f4917bedeac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:05.957876 25796 tablet_bootstrap.cc:492] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Bootstrap starting.
I20260926 20:51:05.961066 25797 tablet_bootstrap.cc:492] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Bootstrap starting.
I20260926 20:51:05.963061 25798 tablet_bootstrap.cc:492] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Bootstrap starting.
I20260926 20:51:05.963395 25796 tablet_bootstrap.cc:654] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:05.964795 25796 log.cc:824] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:05.964923 25797 tablet_bootstrap.cc:654] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:05.966192 25797 log.cc:824] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:05.966535 25798 tablet_bootstrap.cc:654] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:05.967859 25798 log.cc:824] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:05.968379 25796 tablet_bootstrap.cc:492] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: No bootstrap required, opened a new log
I20260926 20:51:05.968709 25796 ts_tablet_manager.cc:1397] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260926 20:51:05.969501 25797 tablet_bootstrap.cc:492] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: No bootstrap required, opened a new log
I20260926 20:51:05.969781 25797 ts_tablet_manager.cc:1397] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260926 20:51:05.970950 25798 tablet_bootstrap.cc:492] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: No bootstrap required, opened a new log
I20260926 20:51:05.971252 25798 ts_tablet_manager.cc:1397] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260926 20:51:05.979031 25798 raft_consensus.cc:348] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:05.979565 25798 raft_consensus.cc:374] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:05.979671 25798 raft_consensus.cc:729] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4939bad2994cabac29092a21ed84c8, State: Initialized, Role: FOLLOWER
I20260926 20:51:05.980403 25798 consensus_queue.cc:260] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:05.981281 25797 raft_consensus.cc:348] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:05.981774 25797 raft_consensus.cc:374] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:05.981879 25797 raft_consensus.cc:729] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efde0739814b4d90b967971ca38234ff, State: Initialized, Role: FOLLOWER
I20260926 20:51:05.982373 25798 ts_tablet_manager.cc:1428] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260926 20:51:05.982679 25797 consensus_queue.cc:260] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:05.982707 25796 raft_consensus.cc:348] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:05.983187 25796 raft_consensus.cc:374] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:05.983850 25647 heartbeater.cc:503] Master 127.5.78.62:35697 was elected leader, sending a full tablet report...
I20260926 20:51:05.984622 25797 ts_tablet_manager.cc:1428] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260926 20:51:05.983306 25796 raft_consensus.cc:729] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c4718482cc4de282846fc6240ae5d8, State: Initialized, Role: FOLLOWER
I20260926 20:51:05.985738 25796 consensus_queue.cc:260] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:05.987128 25777 heartbeater.cc:503] Master 127.5.78.62:35697 was elected leader, sending a full tablet report...
I20260926 20:51:05.987733 25796 ts_tablet_manager.cc:1428] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Time spent starting tablet: real 0.019s user 0.009s sys 0.007s
W20260926 20:51:06.092296 25648 tablet.cc:2369] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:51:06.109509 25517 tablet.cc:2369] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:51:06.110718 25778 tablet.cc:2369] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:51:06.131381 25804 raft_consensus.cc:482] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:51:06.131685 25804 raft_consensus.cc:504] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:06.133492 25804 leader_election.cc:288] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efde0739814b4d90b967971ca38234ff (127.5.78.2:38583), cd4939bad2994cabac29092a21ed84c8 (127.5.78.1:46493)
I20260926 20:51:06.141157 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.133910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:38099 (local address 127.5.78.2:38583)
0926 20:51:06.134446 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:06.134455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:06.134481 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:51:06.134607 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:06.134616 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:06.134713 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:06.134836 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:06.135417 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.136406 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.139033 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.140202 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.140348 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.140464 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:06.140755 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 20:51:06.140895 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260926 20:51:06.141211 25779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.133897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.2:38583 (local address 127.5.78.3:38099)
0926 20:51:06.134151 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.134173 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:06.134318 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:06.134953 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:06.134959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:06.134990 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:06.135236 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:06.135250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.136568 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.136573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:06.138850 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:06.138868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.140405 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.140410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:06.140550 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.140578 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 20:51:06.140712 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 20:51:06.140871 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":30}
I20260926 20:51:06.141456 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.134043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.78.3:34059 (local address 127.5.78.1:46493)
0926 20:51:06.134274 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:06.134280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:06.135163 (+ 883us) server_negotiation.cc:407] Connection header received
0926 20:51:06.135307 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:06.135311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:06.135438 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:06.135544 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:51:06.136015 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.136849 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.139339 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.140575 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.140706 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.140801 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:06.141086 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 20:51:06.141228 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:51:06.141660 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.134154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.5.78.3:34059)
0926 20:51:06.135014 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.135034 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:06.135174 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:06.135564 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:06.135567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:06.135587 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:51:06.135846 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:06.135853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.137008 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.137014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:06.139146 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:06.139157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.140749 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.140754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:06.140907 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.140939 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:51:06.141036 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:51:06.141149 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":49,"thread_start_us":477,"threads_started":1}
I20260926 20:51:06.142472 25602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d0a863b369423f89c7f4917bedeac8" candidate_uuid: "69c4718482cc4de282846fc6240ae5d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efde0739814b4d90b967971ca38234ff" is_pre_election: true
I20260926 20:51:06.142728 25471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d0a863b369423f89c7f4917bedeac8" candidate_uuid: "69c4718482cc4de282846fc6240ae5d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4939bad2994cabac29092a21ed84c8" is_pre_election: true
I20260926 20:51:06.143159 25602 raft_consensus.cc:2454] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c4718482cc4de282846fc6240ae5d8 in term 0.
I20260926 20:51:06.143234 25471 raft_consensus.cc:2454] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c4718482cc4de282846fc6240ae5d8 in term 0.
I20260926 20:51:06.143957 25666 leader_election.cc:302] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 69c4718482cc4de282846fc6240ae5d8, cd4939bad2994cabac29092a21ed84c8, efde0739814b4d90b967971ca38234ff; no voters:
I20260926 20:51:06.144414 25804 raft_consensus.cc:2781] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:51:06.144558 25804 raft_consensus.cc:482] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:51:06.144686 25804 raft_consensus.cc:3037] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:06.147106 25804 raft_consensus.cc:504] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:06.147859 25804 leader_election.cc:288] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [CANDIDATE]: Term 1 election: Requested vote from peers efde0739814b4d90b967971ca38234ff (127.5.78.2:38583), cd4939bad2994cabac29092a21ed84c8 (127.5.78.1:46493)
I20260926 20:51:06.148439 25602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d0a863b369423f89c7f4917bedeac8" candidate_uuid: "69c4718482cc4de282846fc6240ae5d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efde0739814b4d90b967971ca38234ff"
I20260926 20:51:06.148669 25602 raft_consensus.cc:3037] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:06.148633 25471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39d0a863b369423f89c7f4917bedeac8" candidate_uuid: "69c4718482cc4de282846fc6240ae5d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4939bad2994cabac29092a21ed84c8"
I20260926 20:51:06.148875 25471 raft_consensus.cc:3037] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:06.151252 25602 raft_consensus.cc:2454] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c4718482cc4de282846fc6240ae5d8 in term 1.
I20260926 20:51:06.151283 25471 raft_consensus.cc:2454] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c4718482cc4de282846fc6240ae5d8 in term 1.
I20260926 20:51:06.151881 25665 leader_election.cc:302] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c4718482cc4de282846fc6240ae5d8, efde0739814b4d90b967971ca38234ff; no voters:
I20260926 20:51:06.152297 25804 raft_consensus.cc:2781] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:06.152765 25804 raft_consensus.cc:686] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 1 LEADER]: Becoming Leader. State: Replica: 69c4718482cc4de282846fc6240ae5d8, State: Running, Role: LEADER
I20260926 20:51:06.153280 25804 consensus_queue.cc:237] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } }
I20260926 20:51:06.158806 25329 catalog_manager.cc:5495] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c4718482cc4de282846fc6240ae5d8 (127.5.78.3). New cstate: current_term: 1 leader_uuid: "69c4718482cc4de282846fc6240ae5d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c4718482cc4de282846fc6240ae5d8" member_type: VOTER last_known_addr { host: "127.5.78.3" port: 44717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 } health_report { overall_health: UNKNOWN } } }
I20260926 20:51:06.183905 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.177775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51656 (local address 127.5.78.3:44717)
0926 20:51:06.178110 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:06.178115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:06.178132 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:51:06.178242 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:06.178246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:06.178322 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:06.178527 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:06.178973 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.179850 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.181438 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.181916 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.182069 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.182692 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:06.183226 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:06.183401 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:06.183517 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 20:51:06.183604 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260926 20:51:06.192262 25471 raft_consensus.cc:1261] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 1 FOLLOWER]: Refusing update from remote peer 69c4718482cc4de282846fc6240ae5d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:06.192276 25602 raft_consensus.cc:1261] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 1 FOLLOWER]: Refusing update from remote peer 69c4718482cc4de282846fc6240ae5d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:06.193183 25804 consensus_queue.cc:1035] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4939bad2994cabac29092a21ed84c8" member_type: VOTER last_known_addr { host: "127.5.78.1" port: 46493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:06.193509 25806 consensus_queue.cc:1035] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efde0739814b4d90b967971ca38234ff" member_type: VOTER last_known_addr { host: "127.5.78.2" port: 38583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:06.202139 25809 mvcc.cc:205] Tried to move back new op lower bound from 7333707227913064448 to 7333707227767881728. Current Snapshot: MvccSnapshot[applied={T|T < 7333707227913064448}]
I20260926 20:51:06.222550 25330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:51:06.222838 25330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:51:06.226303 25330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf1b9999dc62424082fc96d79201af13: Sending DeleteTablet for 3 replicas of tablet 39d0a863b369423f89c7f4917bedeac8
I20260926 20:51:06.227522 25712 tablet_service.cc:1514] Processing DeleteTablet for tablet 39d0a863b369423f89c7f4917bedeac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:51:06 UTC) from {username='slave'} at 127.0.0.1:51640
I20260926 20:51:06.227516 25582 tablet_service.cc:1514] Processing DeleteTablet for tablet 39d0a863b369423f89c7f4917bedeac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:51:06 UTC) from {username='slave'} at 127.0.0.1:33356
I20260926 20:51:06.227758 25451 tablet_service.cc:1514] Processing DeleteTablet for tablet 39d0a863b369423f89c7f4917bedeac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:51:06 UTC) from {username='slave'} at 127.0.0.1:55186
I20260926 20:51:06.229542 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25389
I20260926 20:51:06.238458 25819 tablet_replica.cc:330] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: stopping tablet replica
I20260926 20:51:06.239118 25819 raft_consensus.cc:2229] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:51:06.239807 25819 raft_consensus.cc:2258] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:51:06.244968 25819 ts_tablet_manager.cc:1905] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:51:06.249480 25820 tablet_replica.cc:330] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: stopping tablet replica
I20260926 20:51:06.250116 25820 raft_consensus.cc:2229] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:51:06.251398 25821 tablet_replica.cc:330] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: stopping tablet replica
I20260926 20:51:06.251799 25821 raft_consensus.cc:2229] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:51:06.252259 25821 raft_consensus.cc:2258] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:51:06.253705 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.230988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55196 (local address 127.5.78.1:46493)
0926 20:51:06.231169 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:06.231176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:06.244011 (+ 12835us) server_negotiation.cc:407] Connection header received
0926 20:51:06.244535 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:06.244542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:06.244678 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:06.244766 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:06.245646 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.246910 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.248860 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.249546 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.249665 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.251868 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:06.251884 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:06.251902 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:06.252726 (+ 824us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:06.252789 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:06.252815 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:06.252900 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:06.253204 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 20:51:06.253335 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:51:06.254452 25821 ts_tablet_manager.cc:1905] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:51:06.255182 25511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:06.259241 25819 ts_tablet_manager.cc:1918] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:51:06.259472 25819 log.cc:1196] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-1/wal/wals/39d0a863b369423f89c7f4917bedeac8
I20260926 20:51:06.260051 25819 ts_tablet_manager.cc:1939] T 39d0a863b369423f89c7f4917bedeac8 P efde0739814b4d90b967971ca38234ff: Deleting consensus metadata
I20260926 20:51:06.261955 25314 catalog_manager.cc:4841] TS efde0739814b4d90b967971ca38234ff (127.5.78.2:38583): tablet 39d0a863b369423f89c7f4917bedeac8 (table test-table [id=739b4ec9e8284260bf4b5947041adb3b]) successfully deleted
I20260926 20:51:06.262768 25820 raft_consensus.cc:2258] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:51:06.264947 25820 ts_tablet_manager.cc:1905] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:51:06.272573 25820 ts_tablet_manager.cc:1918] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:51:06.272773 25820 log.cc:1196] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-2/wal/wals/39d0a863b369423f89c7f4917bedeac8
I20260926 20:51:06.273512 25820 ts_tablet_manager.cc:1939] T 39d0a863b369423f89c7f4917bedeac8 P 69c4718482cc4de282846fc6240ae5d8: Deleting consensus metadata
I20260926 20:51:06.275578 25316 catalog_manager.cc:4841] TS 69c4718482cc4de282846fc6240ae5d8 (127.5.78.3:44717): tablet 39d0a863b369423f89c7f4917bedeac8 (table test-table [id=739b4ec9e8284260bf4b5947041adb3b]) successfully deleted
I20260926 20:51:06.391633 25821 ts_tablet_manager.cc:1918] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:51:06.391791 25821 log.cc:1196] T 39d0a863b369423f89c7f4917bedeac8 P cd4939bad2994cabac29092a21ed84c8: Deleting WAL directory at /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455541382686-5432-0/minicluster-data/ts-0/wal/wals/39d0a863b369423f89c7f4917bedeac8
I20260926 20:51:06.391969 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25389
I20260926 20:51:06.409912 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25520
W20260926 20:51:06.409916 25315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.78.1:46493 (error 108)
I20260926 20:51:06.412151 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.411594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55204)
0926 20:51:06.411843 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.411975 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260926 20:51:06.412406 25315 catalog_manager.cc:4584] TS cd4939bad2994cabac29092a21ed84c8 (127.5.78.1:46493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39d0a863b369423f89c7f4917bedeac8: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
I20260926 20:51:06.417680 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.411179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.5.78.2:38583)
0926 20:51:06.411362 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:06.411366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:06.411382 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:51:06.411439 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:06.411442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:06.411521 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:06.411652 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:06.412083 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.412978 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.414279 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.414780 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.414923 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.416159 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:06.416170 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:06.416185 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:06.416813 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:06.416864 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:06.416885 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:06.417074 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:06.417224 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 20:51:06.417342 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:51:06.418537 25642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:06.466320 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.465746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55212)
0926 20:51:06.466025 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.466141 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260926 20:51:06.467571 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.467140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55214)
0926 20:51:06.467308 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.467406 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 20:51:06.509092 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25520
I20260926 20:51:06.526275 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25650
I20260926 20:51:06.528362 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.527818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55230)
0926 20:51:06.528041 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.528144 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 20:51:06.529605 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.529194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55234)
0926 20:51:06.529366 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.529464 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 20:51:06.535163 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.527149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.5.78.3:44717)
0926 20:51:06.527320 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:06.527325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:06.527483 (+ 158us) server_negotiation.cc:407] Connection header received
0926 20:51:06.527700 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:06.527705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:06.527816 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:06.527935 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:06.528479 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.529663 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.531393 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:06.532215 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:06.532360 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:06.533590 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:06.533601 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:06.533616 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:06.534270 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:06.534322 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:06.534345 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:06.534507 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:06.534683 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 20:51:06.534802 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:51:06.535928 25772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:06.610245 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.609741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55248)
0926 20:51:06.609970 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.610067 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 20:51:06.611583 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.611103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55258)
0926 20:51:06.611349 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.611433 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260926 20:51:06.660187 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25650
I20260926 20:51:06.679840 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25299
I20260926 20:51:06.681150 25360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:06.791443 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:06.790775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.1:46493 (local address 127.0.0.1:55268)
0926 20:51:06.791132 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:06.791229 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.1:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260926 20:51:06.791436 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25299
2026-09-26T20:51:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3055 ms)
[----------] 6 tests from EncryptionPolicyTest (22674 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:06 dist-test-slave-z7j8 krb5kdc[25830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:06 dist-test-slave-z7j8 krb5kdc[25830](info): set up 2 sockets
Sep 26 20:51:06 dist-test-slave-z7j8 krb5kdc[25830](info): commencing operation
krb5kdc: starting...
W20260926 20:51:07.922911 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:08 dist-test-slave-z7j8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:08.187063 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:08.531692 25843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:08.532015 25843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:08.532181 25843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:08.540047 25843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:08.540189 25843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:08.540270 25843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:08.540366 25843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:08.540431 25843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:08.551112 25843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:08.552659 25843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:08.554131 25843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:08.567787 25848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:08.567845 25849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:08.567840 25851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:08.568764 25843 server_base.cc:1034] running on GCE node
I20260926 20:51:08.569736 25843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:08.571578 25843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:08.572921 25843 hybrid_clock.cc:648] HybridClock initialized: now 1790455868572841 us; error 77 us; skew 500 ppm
I20260926 20:51:08.573426 25843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:08 dist-test-slave-z7j8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:08.577152 25843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:08.578929 25843 webserver.cc:460] Webserver started at http://127.0.0.1:42759/ using document root <none> and password file <none>
I20260926 20:51:08.579741 25843 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:08.579900 25843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:08.580236 25843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:08.583770 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "3530dc3225f745939ebbe37c29c46176"
format_stamp: "Formatted at 2026-09-26 20:51:08 on dist-test-slave-z7j8"
server_key: "36904706ba620cd56abbfe73a1b5a5fc"
server_key_iv: "e66f72d28bf58b499d7839e06cd2cbee"
server_key_version: "encryptionkey@0"
I20260926 20:51:08.584671 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "3530dc3225f745939ebbe37c29c46176"
format_stamp: "Formatted at 2026-09-26 20:51:08 on dist-test-slave-z7j8"
server_key: "36904706ba620cd56abbfe73a1b5a5fc"
server_key_iv: "e66f72d28bf58b499d7839e06cd2cbee"
server_key_version: "encryptionkey@0"
I20260926 20:51:08.590377 25843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:51:08.594234 25858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:08.595896 25843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:08.596150 25843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "3530dc3225f745939ebbe37c29c46176"
format_stamp: "Formatted at 2026-09-26 20:51:08 on dist-test-slave-z7j8"
server_key: "36904706ba620cd56abbfe73a1b5a5fc"
server_key_iv: "e66f72d28bf58b499d7839e06cd2cbee"
server_key_version: "encryptionkey@0"
I20260926 20:51:08.596405 25843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:08.620450 25843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:08.625116 25843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:08.625461 25843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:08.645069 25843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39633
I20260926 20:51:08.645119 25909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39633 every 8 connection(s)
I20260926 20:51:08.646688 25843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:08.651050 25910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:08.654454 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25843
I20260926 20:51:08.654707 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:08.655534 5432 external_mini_cluster.cc:1428] Setting key 1cba6d2c904826ff4091d4598b9f8fd6
I20260926 20:51:08.665151 25910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176: Bootstrap starting.
I20260926 20:51:08.669430 25910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:08.670984 25910 log.cc:824] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:51:08 dist-test-slave-z7j8 krb5kdc[25830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:51:08.674664 25910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176: No bootstrap required, opened a new log
I20260926 20:51:08.681864 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:08.657867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43359 (local address 127.0.0.1:39633)
0926 20:51:08.658781 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:08.658794 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:08.658840 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:51:08.660629 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:08.660654 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:08.661141 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:08.661513 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:08.663514 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:08.664977 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:08.666940 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:08.667730 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:08.667889 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:08.673613 (+ 5724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:08.673622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:08.673639 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:08.673681 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:08.677103 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:08.678197 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:08.678204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:08.678211 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:08.678378 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:08.678927 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:08.678930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:08.678932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:08.679468 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:08.679711 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:08.680296 (+ 585us) server_negotiation.cc:299] Negotiation successful
0926 20:51:08.680668 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":413,"threads_started":1}
I20260926 20:51:08.684021 25910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3530dc3225f745939ebbe37c29c46176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39633 } }
I20260926 20:51:08.684482 25910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:08.684618 25910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3530dc3225f745939ebbe37c29c46176, State: Initialized, Role: FOLLOWER
I20260926 20:51:08.685336 25910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3530dc3225f745939ebbe37c29c46176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39633 } }
I20260926 20:51:08.685634 25910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:08.685783 25910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:08.685990 25910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:08.686432 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:08.689376 25910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3530dc3225f745939ebbe37c29c46176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39633 } }
I20260926 20:51:08.690128 25910 leader_election.cc:302] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3530dc3225f745939ebbe37c29c46176; no voters:
I20260926 20:51:08.690940 25910 leader_election.cc:288] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:08.691075 25918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:08.692814 25918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [term 1 LEADER]: Becoming Leader. State: Replica: 3530dc3225f745939ebbe37c29c46176, State: Running, Role: LEADER
I20260926 20:51:08.693612 25918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3530dc3225f745939ebbe37c29c46176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39633 } }
I20260926 20:51:08.694320 25910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:08.699048 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3530dc3225f745939ebbe37c29c46176. Latest consensus state: current_term: 1 leader_uuid: "3530dc3225f745939ebbe37c29c46176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3530dc3225f745939ebbe37c29c46176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39633 } } }
I20260926 20:51:08.699509 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:08.701313 25920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3530dc3225f745939ebbe37c29c46176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3530dc3225f745939ebbe37c29c46176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39633 } } }
I20260926 20:51:08.701719 25920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:08.704473 25925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:08.710691 25925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:08.715651 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:08.692994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38047 (local address 127.0.0.1:39633)
0926 20:51:08.693213 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:08.693219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:08.693250 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:51:08.693436 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:08.693452 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:08.693548 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:08.693682 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:08.695873 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:08.697585 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:08.700378 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:08.701314 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:08.701501 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:08.705899 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:08.705907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:08.705911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:08.705959 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:08.710288 (+ 4329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:08.711758 (+ 1470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:08.711767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:08.711776 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:08.711983 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:08.712746 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:08.712753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:08.712759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:08.713342 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:08.713742 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:08.715127 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0926 20:51:08.715308 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 20:51:08.716251 25937 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:51:08.720835 25925 catalog_manager.cc:1347] Generated new cluster ID: f6fda05fd9834476bc514727c813264f
I20260926 20:51:08.720974 25925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:08.743938 25925 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:08.745359 25925 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:08.751093 25925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3530dc3225f745939ebbe37c29c46176: Generated new TSK 0
I20260926 20:51:08.752218 25925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:08.758682 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25843
I20260926 20:51:08.759953 25905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:08.852121 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25843
W20260926 20:51:08.869704 25828 connection.cc:535] client connection to 127.0.0.1:39633 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39633): Connection reset by peer (error 104)
2026-09-26T20:51:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:08 dist-test-slave-z7j8 krb5kdc[25942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:08 dist-test-slave-z7j8 krb5kdc[25942](info): set up 2 sockets
Sep 26 20:51:08 dist-test-slave-z7j8 krb5kdc[25942](info): commencing operation
krb5kdc: starting...
W20260926 20:51:09.954850 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:10 dist-test-slave-z7j8 krb5kdc[25942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:10.221781 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:10.568005 25955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:10.568387 25955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:10.568570 25955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:10.576253 25955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:10.576440 25955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:10.576552 25955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:10.576671 25955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:10.576750 25955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:10.587349 25955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:10.588752 25955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:10.590185 25955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:10.603099 25960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:10.602988 25961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:10.603039 25963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:10.603976 25955 server_base.cc:1034] running on GCE node
I20260926 20:51:10.604873 25955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:10.606575 25955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:10.607826 25955 hybrid_clock.cc:648] HybridClock initialized: now 1790455870607764 us; error 56 us; skew 500 ppm
I20260926 20:51:10.608211 25955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:10 dist-test-slave-z7j8 krb5kdc[25942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455870, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:10.611586 25955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:10.613306 25955 webserver.cc:460] Webserver started at http://127.0.0.1:41225/ using document root <none> and password file <none>
I20260926 20:51:10.614070 25955 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:10.614228 25955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:10.614609 25955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:10.617753 25955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "81676c49e53c47f2b48456f38477db60"
format_stamp: "Formatted at 2026-09-26 20:51:10 on dist-test-slave-z7j8"
server_key: "99ada727e9fbeab81e391734bd0478d1"
server_key_iv: "e1a79372d1c07836cae63617280ee862"
server_key_version: "encryptionkey@0"
I20260926 20:51:10.618626 25955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "81676c49e53c47f2b48456f38477db60"
format_stamp: "Formatted at 2026-09-26 20:51:10 on dist-test-slave-z7j8"
server_key: "99ada727e9fbeab81e391734bd0478d1"
server_key_iv: "e1a79372d1c07836cae63617280ee862"
server_key_version: "encryptionkey@0"
I20260926 20:51:10.624305 25955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:51:10.628564 25970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:10.630219 25955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:51:10.630471 25955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "81676c49e53c47f2b48456f38477db60"
format_stamp: "Formatted at 2026-09-26 20:51:10 on dist-test-slave-z7j8"
server_key: "99ada727e9fbeab81e391734bd0478d1"
server_key_iv: "e1a79372d1c07836cae63617280ee862"
server_key_version: "encryptionkey@0"
I20260926 20:51:10.630715 25955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:10.660414 25955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:10.665242 25955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:10.665586 25955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:10.685180 25955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33585
I20260926 20:51:10.685213 26021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33585 every 8 connection(s)
I20260926 20:51:10.686802 25955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:10.689419 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 25955
I20260926 20:51:10.689623 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:10.690127 5432 external_mini_cluster.cc:1428] Setting key b3878d0dc3d1c09234133d1e972e52fb
I20260926 20:51:10.691124 26022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:10.703940 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:10.691995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60833 (local address 127.0.0.1:33585)
0926 20:51:10.692878 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:10.692891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:10.692937 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:51:10.694698 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:10.694726 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:10.695207 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:10.695692 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:10.696281 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:10.697711 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:10.699216 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:10.699934 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:10.700144 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:10.701478 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:10.701489 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:10.701504 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:10.702121 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:10.702205 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:10.702230 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:10.702411 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:10.702742 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 20:51:10.702924 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":387,"threads_started":1}
I20260926 20:51:10.704378 26022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60: Bootstrap starting.
I20260926 20:51:10.708210 26022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:10.708521 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:10.709416 26022 log.cc:824] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:10.712881 26022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60: No bootstrap required, opened a new log
I20260926 20:51:10.720770 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:10.713420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51333 (local address 127.0.0.1:33585)
0926 20:51:10.713619 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:10.713631 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:10.714143 (+ 512us) server_negotiation.cc:407] Connection header received
0926 20:51:10.714316 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:10.714321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:10.714401 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:10.714549 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:10.715032 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:10.716246 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:10.717767 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:10.718571 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:10.718722 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:10.719888 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:10.719894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:10.719900 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:10.719930 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:10.719979 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:10.720000 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:10.720135 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:10.720370 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 20:51:10.720438 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:51:10.723102 26022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81676c49e53c47f2b48456f38477db60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260926 20:51:10.723508 26022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:10.723588 26022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81676c49e53c47f2b48456f38477db60, State: Initialized, Role: FOLLOWER
I20260926 20:51:10.724138 26022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81676c49e53c47f2b48456f38477db60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260926 20:51:10.724330 26022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:10.724431 26022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:10.724552 26022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:10.726850 26022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81676c49e53c47f2b48456f38477db60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260926 20:51:10.727383 26022 leader_election.cc:302] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81676c49e53c47f2b48456f38477db60; no voters:
I20260926 20:51:10.727993 26022 leader_election.cc:288] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:10.728164 26032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:10.728881 26032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [term 1 LEADER]: Becoming Leader. State: Replica: 81676c49e53c47f2b48456f38477db60, State: Running, Role: LEADER
I20260926 20:51:10.729413 26032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81676c49e53c47f2b48456f38477db60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } }
I20260926 20:51:10.730126 26022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:10.732558 26033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81676c49e53c47f2b48456f38477db60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81676c49e53c47f2b48456f38477db60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } } }
I20260926 20:51:10.732918 26033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:10.732453 26034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81676c49e53c47f2b48456f38477db60. Latest consensus state: current_term: 1 leader_uuid: "81676c49e53c47f2b48456f38477db60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81676c49e53c47f2b48456f38477db60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33585 } } }
I20260926 20:51:10.733287 26034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:10.735304 26039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:10.740877 26039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:10.751098 26039 catalog_manager.cc:1347] Generated new cluster ID: d31418e4f358443b9a074cc29641859e
I20260926 20:51:10.751252 26039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:10.771106 26039 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:10.772486 26039 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:10.782269 26039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81676c49e53c47f2b48456f38477db60: Generated new TSK 0
I20260926 20:51:10.783257 26039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:10.792806 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25955
W20260926 20:51:10.794289 26017 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60833
W20260926 20:51:10.795162 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 25955: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:10.795361 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 25955
W20260926 20:51:10.812549 25940 connection.cc:535] client connection to 127.0.0.1:33585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33585): Connection reset by peer (error 104)
2026-09-26T20:51:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1942 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:10 dist-test-slave-z7j8 krb5kdc[26053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:10 dist-test-slave-z7j8 krb5kdc[26053](info): set up 2 sockets
Sep 26 20:51:10 dist-test-slave-z7j8 krb5kdc[26053](info): commencing operation
krb5kdc: starting...
W20260926 20:51:11.902813 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:12 dist-test-slave-z7j8 krb5kdc[26053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:12.171433 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:12.536015 26066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:12.536334 26066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:12.536473 26066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:12.543949 26066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:12.544077 26066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:12.544170 26066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:12.544277 26066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:12.544360 26066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:12.554622 26066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:12.556025 26066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:12.557432 26066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:12.569571 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:12.569595 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:12.569527 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:12.570858 26066 server_base.cc:1034] running on GCE node
I20260926 20:51:12.571722 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:12.573220 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:12.574512 26066 hybrid_clock.cc:648] HybridClock initialized: now 1790455872574436 us; error 57 us; skew 500 ppm
I20260926 20:51:12.574924 26066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:12 dist-test-slave-z7j8 krb5kdc[26053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:12.578531 26066 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:12.580106 26066 webserver.cc:460] Webserver started at http://127.0.0.1:45875/ using document root <none> and password file <none>
I20260926 20:51:12.580826 26066 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:12.580955 26066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:12.581283 26066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:12.584386 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "945740cf65824cc09bdbcc702d0f8d2d"
format_stamp: "Formatted at 2026-09-26 20:51:12 on dist-test-slave-z7j8"
server_key: "f1404067cc0b2fe43fbc3df2b4ff64e3"
server_key_iv: "b632c67956e65aef805427ff226f0553"
server_key_version: "encryptionkey@0"
I20260926 20:51:12.585172 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "945740cf65824cc09bdbcc702d0f8d2d"
format_stamp: "Formatted at 2026-09-26 20:51:12 on dist-test-slave-z7j8"
server_key: "f1404067cc0b2fe43fbc3df2b4ff64e3"
server_key_iv: "b632c67956e65aef805427ff226f0553"
server_key_version: "encryptionkey@0"
I20260926 20:51:12.590960 26066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:51:12.594858 26081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:12.596386 26066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:12.596593 26066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "945740cf65824cc09bdbcc702d0f8d2d"
format_stamp: "Formatted at 2026-09-26 20:51:12 on dist-test-slave-z7j8"
server_key: "f1404067cc0b2fe43fbc3df2b4ff64e3"
server_key_iv: "b632c67956e65aef805427ff226f0553"
server_key_version: "encryptionkey@0"
I20260926 20:51:12.596825 26066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:12.602890 26066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:12.607193 26066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:12.607517 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:12.626488 26066 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44447
I20260926 20:51:12.626583 26132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44447 every 8 connection(s)
I20260926 20:51:12.627977 26066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:12.628456 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26066
I20260926 20:51:12.628644 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:12.629305 5432 external_mini_cluster.cc:1428] Setting key db6a6a4de62105ce159617d89ed54ec9
I20260926 20:51:12.632575 26133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:12.637459 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:12.631241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53081 (local address 127.0.0.1:44447)
0926 20:51:12.632074 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:12.632088 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:12.632129 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:51:12.633877 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:12.633901 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:12.634243 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:12.634680 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:12.635805 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:12.635811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:12.635824 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:12.635844 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:12.635915 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:12.635935 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:12.636181 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:12.636321 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 20:51:12.636509 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":339,"threads_started":1}
I20260926 20:51:12.641666 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:12.644944 26133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d: Bootstrap starting.
I20260926 20:51:12.648646 26133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:12.648939 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:12.646217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59599 (local address 127.0.0.1:44447)
0926 20:51:12.646442 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:12.646451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:12.646778 (+ 327us) server_negotiation.cc:407] Connection header received
0926 20:51:12.647004 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:12.647009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:12.647060 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:12.647178 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:12.648224 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:12.648232 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:12.648237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:12.648264 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:12.648316 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:12.648333 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:12.648458 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:12.648593 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 20:51:12.648662 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:51:12.650156 26133 log.cc:824] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:12.653146 26133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d: No bootstrap required, opened a new log
I20260926 20:51:12.660737 26133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945740cf65824cc09bdbcc702d0f8d2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44447 } }
I20260926 20:51:12.661299 26133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:12.661429 26133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945740cf65824cc09bdbcc702d0f8d2d, State: Initialized, Role: FOLLOWER
I20260926 20:51:12.662032 26133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945740cf65824cc09bdbcc702d0f8d2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44447 } }
I20260926 20:51:12.662286 26133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:12.662453 26133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:12.662644 26133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:12.664913 26133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945740cf65824cc09bdbcc702d0f8d2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44447 } }
I20260926 20:51:12.665518 26133 leader_election.cc:302] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 945740cf65824cc09bdbcc702d0f8d2d; no voters:
I20260926 20:51:12.666180 26133 leader_election.cc:288] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:12.666334 26143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:12.667085 26143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [term 1 LEADER]: Becoming Leader. State: Replica: 945740cf65824cc09bdbcc702d0f8d2d, State: Running, Role: LEADER
I20260926 20:51:12.667671 26143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945740cf65824cc09bdbcc702d0f8d2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44447 } }
I20260926 20:51:12.668437 26133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:12.670856 26144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "945740cf65824cc09bdbcc702d0f8d2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945740cf65824cc09bdbcc702d0f8d2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44447 } } }
I20260926 20:51:12.671365 26144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:12.670800 26145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 945740cf65824cc09bdbcc702d0f8d2d. Latest consensus state: current_term: 1 leader_uuid: "945740cf65824cc09bdbcc702d0f8d2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945740cf65824cc09bdbcc702d0f8d2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44447 } } }
I20260926 20:51:12.671850 26145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:12.673023 26152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:12.679112 26152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:12.689316 26152 catalog_manager.cc:1347] Generated new cluster ID: adaa812c00b04eac8313321ca6769f70
I20260926 20:51:12.689481 26152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:12.712456 26152 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:12.713002 26152 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:12.726586 26152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 945740cf65824cc09bdbcc702d0f8d2d: Generated new TSK 0
I20260926 20:51:12.727543 26152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:12.733757 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26066
W20260926 20:51:12.735023 26128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53081
W20260926 20:51:12.735828 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26066: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:12.736025 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26066
2026-09-26T20:51:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:12 dist-test-slave-z7j8 krb5kdc[26164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:12 dist-test-slave-z7j8 krb5kdc[26164](info): set up 2 sockets
Sep 26 20:51:12 dist-test-slave-z7j8 krb5kdc[26164](info): commencing operation
krb5kdc: starting...
W20260926 20:51:13.838747 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:14 dist-test-slave-z7j8 krb5kdc[26164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:14.109158 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:14.446326 26177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:14.446734 26177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:14.446882 26177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:14.454662 26177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:14.454788 26177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:14.454872 26177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:14.454979 26177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:14.455070 26177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:14.465600 26177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:14.467027 26177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:14.468381 26177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:14.480971 26182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:14.480997 26185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:14.481202 26183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:14.481765 26177 server_base.cc:1034] running on GCE node
I20260926 20:51:14.482713 26177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:14.484439 26177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:14.485709 26177 hybrid_clock.cc:648] HybridClock initialized: now 1790455874485669 us; error 66 us; skew 500 ppm
I20260926 20:51:14.486172 26177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:14 dist-test-slave-z7j8 krb5kdc[26164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:14.489769 26177 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:14.491531 26177 webserver.cc:460] Webserver started at http://127.0.0.1:42153/ using document root <none> and password file <none>
I20260926 20:51:14.492260 26177 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:14.492367 26177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:14.492619 26177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:14.496287 26177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "04c93d0efa3f420cb145a00b7f8d224e"
format_stamp: "Formatted at 2026-09-26 20:51:14 on dist-test-slave-z7j8"
server_key: "a39b9c1b8a91f9dabbac1cd1f23e2311"
server_key_iv: "9d5d065b986df4831b94da8fea349d01"
server_key_version: "encryptionkey@0"
I20260926 20:51:14.497300 26177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "04c93d0efa3f420cb145a00b7f8d224e"
format_stamp: "Formatted at 2026-09-26 20:51:14 on dist-test-slave-z7j8"
server_key: "a39b9c1b8a91f9dabbac1cd1f23e2311"
server_key_iv: "9d5d065b986df4831b94da8fea349d01"
server_key_version: "encryptionkey@0"
I20260926 20:51:14.503050 26177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:51:14.507160 26192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:14.508697 26177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:14.508908 26177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "04c93d0efa3f420cb145a00b7f8d224e"
format_stamp: "Formatted at 2026-09-26 20:51:14 on dist-test-slave-z7j8"
server_key: "a39b9c1b8a91f9dabbac1cd1f23e2311"
server_key_iv: "9d5d065b986df4831b94da8fea349d01"
server_key_version: "encryptionkey@0"
I20260926 20:51:14.509147 26177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:14.527114 26177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:14.532101 26177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:14.532451 26177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:14.553309 26177 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41941
I20260926 20:51:14.553351 26243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41941 every 8 connection(s)
I20260926 20:51:14.554936 26177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:14.558928 26244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:14.565048 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26177
I20260926 20:51:14.565267 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:14.565696 5432 external_mini_cluster.cc:1428] Setting key 89b1b631a0bbd3f0918636fbd814093b
I20260926 20:51:14.571249 26244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e: Bootstrap starting.
I20260926 20:51:14.575301 26244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:14.576593 26244 log.cc:824] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:14.580296 26244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e: No bootstrap required, opened a new log
Sep 26 20:51:14 dist-test-slave-z7j8 krb5kdc[26164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:51:14.589540 26244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c93d0efa3f420cb145a00b7f8d224e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260926 20:51:14.590045 26244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:14.590170 26244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c93d0efa3f420cb145a00b7f8d224e, State: Initialized, Role: FOLLOWER
I20260926 20:51:14.590922 26244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c93d0efa3f420cb145a00b7f8d224e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260926 20:51:14.591180 26244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:14.591352 26244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:14.591535 26244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:14.592178 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:14.567737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46741 (local address 127.0.0.1:41941)
0926 20:51:14.568638 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:14.568657 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:14.568712 (+ 55us) server_negotiation.cc:407] Connection header received
0926 20:51:14.571185 (+ 2473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:14.571222 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:14.571847 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:14.572372 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:14.573994 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:14.576083 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:14.577883 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:14.579002 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:14.579216 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:14.583851 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:14.583866 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:14.583888 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:14.583929 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:14.587401 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:14.588378 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:14.588384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:14.588391 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:14.588605 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:14.589015 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:14.589018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:14.589020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:14.589553 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:14.589785 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:14.590400 (+ 615us) server_negotiation.cc:299] Negotiation successful
0926 20:51:14.590856 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":369,"threads_started":1}
I20260926 20:51:14.594017 26244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c93d0efa3f420cb145a00b7f8d224e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260926 20:51:14.594688 26244 leader_election.cc:302] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04c93d0efa3f420cb145a00b7f8d224e; no voters:
I20260926 20:51:14.595466 26244 leader_election.cc:288] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:14.595650 26249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:14.596554 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:14.597638 26244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:14.606467 26249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [term 1 LEADER]: Becoming Leader. State: Replica: 04c93d0efa3f420cb145a00b7f8d224e, State: Running, Role: LEADER
I20260926 20:51:14.607402 26249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c93d0efa3f420cb145a00b7f8d224e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260926 20:51:14.610394 26251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04c93d0efa3f420cb145a00b7f8d224e. Latest consensus state: current_term: 1 leader_uuid: "04c93d0efa3f420cb145a00b7f8d224e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c93d0efa3f420cb145a00b7f8d224e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } } }
I20260926 20:51:14.610852 26251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:14.611418 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04c93d0efa3f420cb145a00b7f8d224e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c93d0efa3f420cb145a00b7f8d224e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } } }
I20260926 20:51:14.611758 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:14.630594 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:14.603876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57763 (local address 127.0.0.1:41941)
0926 20:51:14.604127 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:14.604136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:14.606137 (+ 2001us) server_negotiation.cc:407] Connection header received
0926 20:51:14.607124 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:14.607133 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:14.607262 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:14.607389 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:14.609912 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:14.611528 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:14.614399 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:14.615336 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:14.615482 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:14.620048 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:14.620060 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:14.620068 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:14.620135 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:14.624886 (+ 4751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:14.626323 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:14.626331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:14.626337 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:14.626589 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:14.627201 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:14.627209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:14.627215 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:14.627892 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:14.629653 (+ 1761us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:14.629845 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 20:51:14.630121 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:51:14.630604 26265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:14.635815 26265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:14.646319 26265 catalog_manager.cc:1347] Generated new cluster ID: 1f75c1b3f51b4efb97c7280ce2df2349
I20260926 20:51:14.646479 26265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:14.681083 26265 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:14.683189 26265 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:14.699077 26265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04c93d0efa3f420cb145a00b7f8d224e: Generated new TSK 0
I20260926 20:51:14.700297 26265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:14.718137 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26177
I20260926 20:51:14.719376 26239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:14.813817 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26177
2026-09-26T20:51:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2077 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:14 dist-test-slave-z7j8 krb5kdc[26277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:14 dist-test-slave-z7j8 krb5kdc[26277](info): set up 2 sockets
Sep 26 20:51:14 dist-test-slave-z7j8 krb5kdc[26277](info): commencing operation
krb5kdc: starting...
W20260926 20:51:15.922791 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:16 dist-test-slave-z7j8 krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:16.186455 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:16.520746 26290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:16.521039 26290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:16.521132 26290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:16.528820 26290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:16.528971 26290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:16.529021 26290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:16.529085 26290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:16.529131 26290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:16.539394 26290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:16.540735 26290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:16.542102 26290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:16.554817 26295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:16.554764 26298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:16.554798 26296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:16.555996 26290 server_base.cc:1034] running on GCE node
I20260926 20:51:16.556847 26290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:16.558449 26290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:16.559731 26290 hybrid_clock.cc:648] HybridClock initialized: now 1790455876559666 us; error 57 us; skew 500 ppm
I20260926 20:51:16.560151 26290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:16 dist-test-slave-z7j8 krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:16.563671 26290 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:16.565316 26290 webserver.cc:460] Webserver started at http://127.0.0.1:43779/ using document root <none> and password file <none>
I20260926 20:51:16.566080 26290 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:16.566233 26290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:16.566623 26290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:16.569779 26290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "6a51cbf30f0d441f8c627e89e6430907"
format_stamp: "Formatted at 2026-09-26 20:51:16 on dist-test-slave-z7j8"
server_key: "cd5d8e9b913e0485f26ac5c218f410f5"
server_key_iv: "7e15f8e5278229fe6ae4caea9afebfa5"
server_key_version: "encryptionkey@0"
I20260926 20:51:16.570715 26290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "6a51cbf30f0d441f8c627e89e6430907"
format_stamp: "Formatted at 2026-09-26 20:51:16 on dist-test-slave-z7j8"
server_key: "cd5d8e9b913e0485f26ac5c218f410f5"
server_key_iv: "7e15f8e5278229fe6ae4caea9afebfa5"
server_key_version: "encryptionkey@0"
I20260926 20:51:16.576326 26290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:51:16.580371 26305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:16.582111 26290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:51:16.582310 26290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "6a51cbf30f0d441f8c627e89e6430907"
format_stamp: "Formatted at 2026-09-26 20:51:16 on dist-test-slave-z7j8"
server_key: "cd5d8e9b913e0485f26ac5c218f410f5"
server_key_iv: "7e15f8e5278229fe6ae4caea9afebfa5"
server_key_version: "encryptionkey@0"
I20260926 20:51:16.582556 26290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:16.604887 26290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:16.609280 26290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:16.609591 26290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:16.629158 26290 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37911
I20260926 20:51:16.629187 26356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37911 every 8 connection(s)
I20260926 20:51:16.630640 26290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:16.633970 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26290
I20260926 20:51:16.634171 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:16.634773 5432 external_mini_cluster.cc:1428] Setting key e777a4b1bb142eafd840efe832de3adf
I20260926 20:51:16.635002 26357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:16.649983 26357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907: Bootstrap starting.
I20260926 20:51:16.650260 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:16.636688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37393 (local address 127.0.0.1:37911)
0926 20:51:16.637555 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:16.637568 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:16.637611 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:51:16.640024 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:16.640051 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:16.640541 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:16.640955 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:16.641650 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:16.643186 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:16.645175 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:16.645943 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:16.646100 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:16.647549 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:16.647559 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:16.647592 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:16.648215 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:16.648300 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:16.648327 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:16.648444 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:16.648743 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 20:51:16.649115 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":413,"threads_started":1}
I20260926 20:51:16.653898 26357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:16.654320 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:16.655113 26357 log.cc:824] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:16.658114 26357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907: No bootstrap required, opened a new log
I20260926 20:51:16.665676 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:16.658740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49073 (local address 127.0.0.1:37911)
0926 20:51:16.658937 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:16.658945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:16.659502 (+ 557us) server_negotiation.cc:407] Connection header received
0926 20:51:16.659659 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:16.659664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:16.659747 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:16.659868 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:16.660334 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:16.661473 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:16.662887 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:16.663640 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:16.663774 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:16.664872 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:16.664889 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:16.664895 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:16.664919 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:16.664973 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:16.665000 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:16.665105 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:16.665225 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 20:51:16.665318 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:51:16.668087 26357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a51cbf30f0d441f8c627e89e6430907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37911 } }
I20260926 20:51:16.668509 26357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:16.668629 26357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a51cbf30f0d441f8c627e89e6430907, State: Initialized, Role: FOLLOWER
I20260926 20:51:16.669294 26357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a51cbf30f0d441f8c627e89e6430907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37911 } }
I20260926 20:51:16.669515 26357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:16.669667 26357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:16.669840 26357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:16.672174 26357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a51cbf30f0d441f8c627e89e6430907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37911 } }
I20260926 20:51:16.672722 26357 leader_election.cc:302] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a51cbf30f0d441f8c627e89e6430907; no voters:
I20260926 20:51:16.673467 26357 leader_election.cc:288] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:16.673592 26367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:16.674420 26367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [term 1 LEADER]: Becoming Leader. State: Replica: 6a51cbf30f0d441f8c627e89e6430907, State: Running, Role: LEADER
I20260926 20:51:16.675091 26367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a51cbf30f0d441f8c627e89e6430907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37911 } }
I20260926 20:51:16.676024 26357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:16.678093 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a51cbf30f0d441f8c627e89e6430907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a51cbf30f0d441f8c627e89e6430907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37911 } } }
I20260926 20:51:16.678542 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:16.679565 26369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a51cbf30f0d441f8c627e89e6430907. Latest consensus state: current_term: 1 leader_uuid: "6a51cbf30f0d441f8c627e89e6430907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a51cbf30f0d441f8c627e89e6430907" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37911 } } }
I20260926 20:51:16.680037 26369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:16.682864 26372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:16.690229 26372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:16.699841 26372 catalog_manager.cc:1347] Generated new cluster ID: 9ee8c173f50c472c9d7182b16db9ec08
I20260926 20:51:16.700033 26372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:16.711292 26372 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:16.712694 26372 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:16.720494 26372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a51cbf30f0d441f8c627e89e6430907: Generated new TSK 0
I20260926 20:51:16.721451 26372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:16.738358 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26290
W20260926 20:51:16.739768 26352 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37393
W20260926 20:51:16.740527 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26290: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:16.740697 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26290
2026-09-26T20:51:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1927 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:16 dist-test-slave-z7j8 krb5kdc[26388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:16 dist-test-slave-z7j8 krb5kdc[26388](info): set up 2 sockets
Sep 26 20:51:16 dist-test-slave-z7j8 krb5kdc[26388](info): commencing operation
krb5kdc: starting...
W20260926 20:51:17.862928 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:18 dist-test-slave-z7j8 krb5kdc[26388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:18.134066 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:18.475255 26401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:18.475562 26401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:18.475656 26401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:18.483177 26401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:18.483317 26401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:18.483364 26401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:18.483431 26401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:18.483479 26401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:18.493546 26401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:18.494936 26401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:18.496199 26401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:18.508592 26406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:18.508526 26409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:18.508558 26407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:18.509392 26401 server_base.cc:1034] running on GCE node
I20260926 20:51:18.510243 26401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:18.511831 26401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:18.513103 26401 hybrid_clock.cc:648] HybridClock initialized: now 1790455878513061 us; error 46 us; skew 500 ppm
I20260926 20:51:18.513527 26401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:18 dist-test-slave-z7j8 krb5kdc[26388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:18.520766 26401 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:18.522619 26401 webserver.cc:460] Webserver started at http://127.0.0.1:38679/ using document root <none> and password file <none>
I20260926 20:51:18.523484 26401 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:18.523659 26401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:18.524014 26401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:18.527743 26401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "10a467d6f7c2438eb219d7935f8503fa"
format_stamp: "Formatted at 2026-09-26 20:51:18 on dist-test-slave-z7j8"
server_key: "ebcc438f936a464f2963d721a80cec1c"
server_key_iv: "92cff00c1b728a5053b4ebe750b75ba9"
server_key_version: "encryptionkey@0"
I20260926 20:51:18.528717 26401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "10a467d6f7c2438eb219d7935f8503fa"
format_stamp: "Formatted at 2026-09-26 20:51:18 on dist-test-slave-z7j8"
server_key: "ebcc438f936a464f2963d721a80cec1c"
server_key_iv: "92cff00c1b728a5053b4ebe750b75ba9"
server_key_version: "encryptionkey@0"
I20260926 20:51:18.534890 26401 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:51:18.538897 26416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:18.540549 26401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:18.540747 26401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "10a467d6f7c2438eb219d7935f8503fa"
format_stamp: "Formatted at 2026-09-26 20:51:18 on dist-test-slave-z7j8"
server_key: "ebcc438f936a464f2963d721a80cec1c"
server_key_iv: "92cff00c1b728a5053b4ebe750b75ba9"
server_key_version: "encryptionkey@0"
I20260926 20:51:18.540966 26401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:18.547334 26401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:18.552182 26401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:18.552510 26401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:18.572716 26401 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33091
I20260926 20:51:18.572777 26467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33091 every 8 connection(s)
I20260926 20:51:18.574276 26401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:18.578961 26468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:18.580910 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26401
I20260926 20:51:18.581112 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:18.581712 5432 external_mini_cluster.cc:1428] Setting key c1e669a5b9406c650349fd0b8226c636
I20260926 20:51:18.590705 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:18.583918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36613 (local address 127.0.0.1:33091)
0926 20:51:18.584875 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:18.584889 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:18.584938 (+ 49us) server_negotiation.cc:407] Connection header received
0926 20:51:18.586773 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:18.586805 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:18.587240 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:18.587704 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:18.588919 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:18.588926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:18.588942 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:18.588957 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:18.589035 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:18.589063 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:18.589314 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:18.589461 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 20:51:18.589664 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":437,"threads_started":1}
I20260926 20:51:18.594947 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:18.595007 26468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa: Bootstrap starting.
I20260926 20:51:18.598587 26468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:18.599926 26468 log.cc:824] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:18.602306 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:18.599460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59273 (local address 127.0.0.1:33091)
0926 20:51:18.599697 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:18.599705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:18.600147 (+ 442us) server_negotiation.cc:407] Connection header received
0926 20:51:18.600301 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:18.600307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:18.600377 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:18.600522 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:18.601579 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:18.601585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:18.601589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:18.601609 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:18.601684 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:18.601707 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:18.601833 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:18.601955 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 20:51:18.602016 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:51:18.603615 26468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa: No bootstrap required, opened a new log
I20260926 20:51:18.612239 26468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a467d6f7c2438eb219d7935f8503fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33091 } }
I20260926 20:51:18.612733 26468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:18.612870 26468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a467d6f7c2438eb219d7935f8503fa, State: Initialized, Role: FOLLOWER
I20260926 20:51:18.613540 26468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a467d6f7c2438eb219d7935f8503fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33091 } }
I20260926 20:51:18.613816 26468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:18.613983 26468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:18.614200 26468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:18.616866 26468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a467d6f7c2438eb219d7935f8503fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33091 } }
I20260926 20:51:18.617585 26468 leader_election.cc:302] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10a467d6f7c2438eb219d7935f8503fa; no voters:
I20260926 20:51:18.618305 26468 leader_election.cc:288] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:18.618460 26478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:18.619225 26478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [term 1 LEADER]: Becoming Leader. State: Replica: 10a467d6f7c2438eb219d7935f8503fa, State: Running, Role: LEADER
I20260926 20:51:18.619798 26478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a467d6f7c2438eb219d7935f8503fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33091 } }
I20260926 20:51:18.620659 26468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:18.622620 26479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10a467d6f7c2438eb219d7935f8503fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a467d6f7c2438eb219d7935f8503fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33091 } } }
I20260926 20:51:18.623121 26479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:18.622745 26480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10a467d6f7c2438eb219d7935f8503fa. Latest consensus state: current_term: 1 leader_uuid: "10a467d6f7c2438eb219d7935f8503fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a467d6f7c2438eb219d7935f8503fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33091 } } }
I20260926 20:51:18.623363 26480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:18.625170 26483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:18.629854 26483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:18.641600 26483 catalog_manager.cc:1347] Generated new cluster ID: c30ceef5285d472a9d9651dce4bdab78
I20260926 20:51:18.641767 26483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:18.667049 26483 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:18.667671 26483 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:18.682351 26483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10a467d6f7c2438eb219d7935f8503fa: Generated new TSK 0
I20260926 20:51:18.683667 26483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:18.694235 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26401
W20260926 20:51:18.695574 26463 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36613
W20260926 20:51:18.696383 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26401: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:18.696554 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26401
2026-09-26T20:51:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1956 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:18 dist-test-slave-z7j8 krb5kdc[26499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:18 dist-test-slave-z7j8 krb5kdc[26499](info): set up 2 sockets
Sep 26 20:51:18 dist-test-slave-z7j8 krb5kdc[26499](info): commencing operation
krb5kdc: starting...
W20260926 20:51:19.822901 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:19 dist-test-slave-z7j8 krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:20.097522 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:20.443425 26512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:20.443758 26512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:20.443915 26512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:20.451431 26512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:20.451570 26512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:20.451658 26512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:20.451776 26512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:20.451844 26512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:20.462129 26512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:20.463558 26512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:20.464919 26512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:20.478113 26517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:20.478188 26518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:20.478140 26520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:20.478843 26512 server_base.cc:1034] running on GCE node
I20260926 20:51:20.479888 26512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:20.481392 26512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:20.482667 26512 hybrid_clock.cc:648] HybridClock initialized: now 1790455880482562 us; error 41 us; skew 500 ppm
I20260926 20:51:20.483109 26512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:20 dist-test-slave-z7j8 krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:20.487092 26512 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:20.488862 26512 webserver.cc:460] Webserver started at http://127.0.0.1:33169/ using document root <none> and password file <none>
I20260926 20:51:20.489666 26512 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:20.489779 26512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:20.490150 26512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:20.493454 26512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "53bfbd5be63041699ee9d230aeb16646"
format_stamp: "Formatted at 2026-09-26 20:51:20 on dist-test-slave-z7j8"
server_key: "e6ce549eaed2b287e42797f7de2e0c0d"
server_key_iv: "d15376285f73f8006406e53c7ece1f62"
server_key_version: "encryptionkey@0"
I20260926 20:51:20.494307 26512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "53bfbd5be63041699ee9d230aeb16646"
format_stamp: "Formatted at 2026-09-26 20:51:20 on dist-test-slave-z7j8"
server_key: "e6ce549eaed2b287e42797f7de2e0c0d"
server_key_iv: "d15376285f73f8006406e53c7ece1f62"
server_key_version: "encryptionkey@0"
I20260926 20:51:20.500011 26512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:51:20.503818 26527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:20.505662 26512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:20.505873 26512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "53bfbd5be63041699ee9d230aeb16646"
format_stamp: "Formatted at 2026-09-26 20:51:20 on dist-test-slave-z7j8"
server_key: "e6ce549eaed2b287e42797f7de2e0c0d"
server_key_iv: "d15376285f73f8006406e53c7ece1f62"
server_key_version: "encryptionkey@0"
I20260926 20:51:20.506085 26512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:20.524950 26512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:20.529755 26512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:20.530088 26512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:20.550575 26512 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46237
I20260926 20:51:20.550575 26578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46237 every 8 connection(s)
I20260926 20:51:20.552153 26512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:20.553786 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26512
I20260926 20:51:20.553985 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:20.554605 5432 external_mini_cluster.cc:1428] Setting key cce47eb484f898adce0dbdddf4042627
I20260926 20:51:20.557986 26579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:51:20 dist-test-slave-z7j8 krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:51:20.570859 26579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646: Bootstrap starting.
I20260926 20:51:20.575157 26579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:20.576624 26579 log.cc:824] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:20.580725 26579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646: No bootstrap required, opened a new log
I20260926 20:51:20.581466 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:20.556786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:46651 (local address 127.0.0.1:46237)
0926 20:51:20.558014 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:20.558030 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:20.558208 (+ 178us) server_negotiation.cc:407] Connection header received
0926 20:51:20.560346 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:20.560375 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:20.560865 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:20.561403 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:20.563130 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:20.564610 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:20.566023 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:20.566862 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:20.567103 (+ 241us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:20.572058 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:20.572069 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:20.572086 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:20.572141 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:20.575901 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:20.577202 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:20.577222 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:20.577229 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:20.577586 (+ 357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:20.578172 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:20.578178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:20.578181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:20.578906 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:20.579162 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:20.579858 (+ 696us) server_negotiation.cc:299] Negotiation successful
0926 20:51:20.580240 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":455,"threads_started":1}
I20260926 20:51:20.586956 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:20.592303 26579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bfbd5be63041699ee9d230aeb16646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260926 20:51:20.592865 26579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:20.593004 26579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bfbd5be63041699ee9d230aeb16646, State: Initialized, Role: FOLLOWER
I20260926 20:51:20.593833 26579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bfbd5be63041699ee9d230aeb16646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260926 20:51:20.594187 26579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:20.594344 26579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:20.594583 26579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:20.597643 26579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bfbd5be63041699ee9d230aeb16646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260926 20:51:20.598341 26579 leader_election.cc:302] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53bfbd5be63041699ee9d230aeb16646; no voters:
I20260926 20:51:20.599184 26579 leader_election.cc:288] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:20.599327 26589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:20.599983 26589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [term 1 LEADER]: Becoming Leader. State: Replica: 53bfbd5be63041699ee9d230aeb16646, State: Running, Role: LEADER
I20260926 20:51:20.600744 26589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bfbd5be63041699ee9d230aeb16646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260926 20:51:20.603569 26590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53bfbd5be63041699ee9d230aeb16646. Latest consensus state: current_term: 1 leader_uuid: "53bfbd5be63041699ee9d230aeb16646" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bfbd5be63041699ee9d230aeb16646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } } }
I20260926 20:51:20.604027 26590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:20.604817 26579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:20.607710 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53bfbd5be63041699ee9d230aeb16646" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bfbd5be63041699ee9d230aeb16646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } } }
I20260926 20:51:20.608117 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:20.611121 26593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:20.612457 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:20.592631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:57775 (local address 127.0.0.1:46237)
0926 20:51:20.592843 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:20.592854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:20.593486 (+ 632us) server_negotiation.cc:407] Connection header received
0926 20:51:20.593718 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:20.593722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:20.593799 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:20.593898 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:20.595953 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:20.596823 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:20.598703 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:20.599209 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:20.599377 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:20.602326 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:20.602332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:20.602337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:20.602387 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:20.606963 (+ 4576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:20.608600 (+ 1637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:20.608609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:20.608612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:20.608822 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:20.609646 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:20.609654 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:20.609659 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:20.610151 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:20.610633 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:20.611739 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0926 20:51:20.611949 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:51:20.617698 26593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:20.627270 26593 catalog_manager.cc:1347] Generated new cluster ID: 109e52d44e944363993446a64f2ef857
I20260926 20:51:20.627409 26593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:20.696244 26593 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:20.698199 26593 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:20.723807 26593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53bfbd5be63041699ee9d230aeb16646: Generated new TSK 0
I20260926 20:51:20.724788 26593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:20.732434 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26512
I20260926 20:51:20.734640 26574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:20.833393 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26512
W20260926 20:51:20.851172 26497 connection.cc:535] client connection to 127.0.0.1:46237 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46237): Connection reset by peer (error 104)
2026-09-26T20:51:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2135 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:20 dist-test-slave-z7j8 krb5kdc[26611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:20 dist-test-slave-z7j8 krb5kdc[26611](info): set up 2 sockets
Sep 26 20:51:20 dist-test-slave-z7j8 krb5kdc[26611](info): commencing operation
krb5kdc: starting...
W20260926 20:51:21.954941 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:22 dist-test-slave-z7j8 krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:22.221829 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:22.581317 26624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:22.581719 26624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:22.581895 26624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:22.589449 26624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:22.589633 26624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:22.589756 26624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:22.589864 26624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:22.589943 26624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:22.600343 26624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:22.601790 26624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:22.603261 26624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:22.615695 26629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:22.615780 26632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:22.615739 26630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:22.617100 26624 server_base.cc:1034] running on GCE node
I20260926 20:51:22.617971 26624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:22.619542 26624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:22.620761 26624 hybrid_clock.cc:648] HybridClock initialized: now 1790455882620715 us; error 35 us; skew 500 ppm
I20260926 20:51:22.621183 26624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:22 dist-test-slave-z7j8 krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:22.624589 26624 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:22.626292 26624 webserver.cc:460] Webserver started at http://127.0.0.1:43161/ using document root <none> and password file <none>
I20260926 20:51:22.627106 26624 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:22.627215 26624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:22.627573 26624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:22.630877 26624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "6c54306a73ea4ece88ee7eab0a76e5a4"
format_stamp: "Formatted at 2026-09-26 20:51:22 on dist-test-slave-z7j8"
server_key: "91491fc1f785cc6a835c2e0a62327f49"
server_key_iv: "2f040fa5af636e4a081244fc9b11e83c"
server_key_version: "encryptionkey@0"
I20260926 20:51:22.631697 26624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "6c54306a73ea4ece88ee7eab0a76e5a4"
format_stamp: "Formatted at 2026-09-26 20:51:22 on dist-test-slave-z7j8"
server_key: "91491fc1f785cc6a835c2e0a62327f49"
server_key_iv: "2f040fa5af636e4a081244fc9b11e83c"
server_key_version: "encryptionkey@0"
I20260926 20:51:22.637454 26624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260926 20:51:22.641402 26639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:22.643067 26624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:51:22.643281 26624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "6c54306a73ea4ece88ee7eab0a76e5a4"
format_stamp: "Formatted at 2026-09-26 20:51:22 on dist-test-slave-z7j8"
server_key: "91491fc1f785cc6a835c2e0a62327f49"
server_key_iv: "2f040fa5af636e4a081244fc9b11e83c"
server_key_version: "encryptionkey@0"
I20260926 20:51:22.643492 26624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:22.661360 26624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:22.665650 26624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:22.665949 26624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:22.685355 26624 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44585
I20260926 20:51:22.685403 26690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44585 every 8 connection(s)
I20260926 20:51:22.686801 26624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:22.687801 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26624
I20260926 20:51:22.688004 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:22.688594 5432 external_mini_cluster.cc:1428] Setting key bb6335ebddafe640a976042048185563
I20260926 20:51:22.691664 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:22.701540 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:22.690484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:47461 (local address 127.0.0.1:44585)
0926 20:51:22.691226 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:22.691237 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:22.691277 (+ 40us) server_negotiation.cc:407] Connection header received
0926 20:51:22.692974 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:22.692996 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:22.693373 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:22.693739 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:22.694292 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:22.695656 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:22.696984 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:22.697631 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:22.697832 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:22.699180 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:22.699188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:22.699202 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:22.699786 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:22.699870 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:22.699899 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:22.700108 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:22.700384 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 20:51:22.700568 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":309,"threads_started":1}
I20260926 20:51:22.702795 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4: Bootstrap starting.
I20260926 20:51:22.705627 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:22.706360 26691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:22.707440 26691 log.cc:824] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:22.710887 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4: No bootstrap required, opened a new log
I20260926 20:51:22.716670 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:22.710152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:55651 (local address 127.0.0.1:44585)
0926 20:51:22.710301 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:22.710305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:22.710844 (+ 539us) server_negotiation.cc:407] Connection header received
0926 20:51:22.711084 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:22.711089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:22.711147 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:22.711249 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:22.711718 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:22.712505 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:22.713934 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:22.714497 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:22.714627 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:22.715899 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:22.715904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:22.715907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:22.715926 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:22.715971 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:22.715994 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:22.716125 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:22.716323 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 20:51:22.716380 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 20:51:22.719453 26691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260926 20:51:22.719877 26691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:22.719990 26691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c54306a73ea4ece88ee7eab0a76e5a4, State: Initialized, Role: FOLLOWER
I20260926 20:51:22.720599 26691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260926 20:51:22.720852 26691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:22.721032 26691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:22.721218 26691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:22.723640 26691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260926 20:51:22.724215 26691 leader_election.cc:302] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c54306a73ea4ece88ee7eab0a76e5a4; no voters:
I20260926 20:51:22.724902 26691 leader_election.cc:288] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:22.724995 26701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:22.725749 26701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [term 1 LEADER]: Becoming Leader. State: Replica: 6c54306a73ea4ece88ee7eab0a76e5a4, State: Running, Role: LEADER
I20260926 20:51:22.726300 26701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260926 20:51:22.727339 26691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:22.729370 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c54306a73ea4ece88ee7eab0a76e5a4. Latest consensus state: current_term: 1 leader_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } } }
I20260926 20:51:22.729858 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:22.731163 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c54306a73ea4ece88ee7eab0a76e5a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } } }
I20260926 20:51:22.731536 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:22.732020 26710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:22.737215 26710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:22.748870 26710 catalog_manager.cc:1347] Generated new cluster ID: 19f47ed3d76945dd875fb13e0b662d27
I20260926 20:51:22.748996 26710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:22.766598 26710 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:22.767922 26710 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:22.782157 26710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c54306a73ea4ece88ee7eab0a76e5a4: Generated new TSK 0
I20260926 20:51:22.783380 26710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:22.798419 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26624
W20260926 20:51:22.800117 26686 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:47461
W20260926 20:51:22.800904 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26624: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:22.801054 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26624
W20260926 20:51:22.818146 26609 connection.cc:535] client connection to 127.0.0.1:44585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44585): Connection reset by peer (error 104)
2026-09-26T20:51:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1966 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:22 dist-test-slave-z7j8 krb5kdc[26722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:22 dist-test-slave-z7j8 krb5kdc[26722](info): set up 2 sockets
Sep 26 20:51:22 dist-test-slave-z7j8 krb5kdc[26722](info): commencing operation
krb5kdc: starting...
W20260926 20:51:23.906899 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:24 dist-test-slave-z7j8 krb5kdc[26722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:24.167747 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:24.508704 26735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:24.509027 26735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:24.509132 26735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:24.516472 26735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:24.516604 26735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:24.516654 26735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:24.516718 26735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:24.516762 26735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:24.526898 26735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:24.528175 26735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:24.529523 26735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:24.542718 26740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:24.542868 26741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:24.543313 26743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:24.543759 26735 server_base.cc:1034] running on GCE node
I20260926 20:51:24.544631 26735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:24.546299 26735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:24.547554 26735 hybrid_clock.cc:648] HybridClock initialized: now 1790455884547508 us; error 40 us; skew 500 ppm
I20260926 20:51:24.547947 26735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:24 dist-test-slave-z7j8 krb5kdc[26722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:24.559952 26735 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:24.561887 26735 webserver.cc:460] Webserver started at http://127.0.0.1:32993/ using document root <none> and password file <none>
I20260926 20:51:24.562773 26735 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:24.562915 26735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:24.563226 26735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:24.566639 26735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "d416b720fa8a48d2a3d9cfb339b2fc68"
format_stamp: "Formatted at 2026-09-26 20:51:24 on dist-test-slave-z7j8"
server_key: "9fa5142afe310aca433b21a1a117997c"
server_key_iv: "9449aa45c1f2b8cae47df002c1c88d0a"
server_key_version: "encryptionkey@0"
I20260926 20:51:24.567538 26735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "d416b720fa8a48d2a3d9cfb339b2fc68"
format_stamp: "Formatted at 2026-09-26 20:51:24 on dist-test-slave-z7j8"
server_key: "9fa5142afe310aca433b21a1a117997c"
server_key_iv: "9449aa45c1f2b8cae47df002c1c88d0a"
server_key_version: "encryptionkey@0"
I20260926 20:51:24.573870 26735 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 20:51:24.578155 26750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:24.579869 26735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:24.580107 26735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "d416b720fa8a48d2a3d9cfb339b2fc68"
format_stamp: "Formatted at 2026-09-26 20:51:24 on dist-test-slave-z7j8"
server_key: "9fa5142afe310aca433b21a1a117997c"
server_key_iv: "9449aa45c1f2b8cae47df002c1c88d0a"
server_key_version: "encryptionkey@0"
I20260926 20:51:24.580359 26735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:24.586849 26735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:24.591676 26735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:24.592010 26735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:24.611398 26735 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38395
I20260926 20:51:24.611454 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38395 every 8 connection(s)
I20260926 20:51:24.612864 26735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:24.613456 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26735
I20260926 20:51:24.613722 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:24.614342 5432 external_mini_cluster.cc:1428] Setting key b58f3e00d41b20e069110b8b8b3db356
I20260926 20:51:24.617492 26802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:24.622828 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:24.616342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:34535 (local address 127.0.0.1:38395)
0926 20:51:24.617175 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:24.617189 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:24.617235 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:51:24.619061 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:24.619088 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:24.619474 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:24.619919 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:24.621053 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:24.621063 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:24.621085 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:24.621111 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:24.621186 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:24.621210 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:24.621393 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:24.621508 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 20:51:24.621749 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":367,"threads_started":1}
I20260926 20:51:24.627298 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:24.630342 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68: Bootstrap starting.
I20260926 20:51:24.633973 26802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:24.634490 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:24.631758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:46683 (local address 127.0.0.1:38395)
0926 20:51:24.631936 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:24.631944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:24.632364 (+ 420us) server_negotiation.cc:407] Connection header received
0926 20:51:24.632576 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:24.632588 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:24.632644 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:24.632747 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:24.633702 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:24.633708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:24.633712 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:24.633731 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:24.633780 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:24.633796 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:24.633919 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:24.634049 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 20:51:24.634118 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 20:51:24.635502 26802 log.cc:824] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:24.638623 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68: No bootstrap required, opened a new log
I20260926 20:51:24.646370 26802 raft_consensus.cc:348] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38395 } }
I20260926 20:51:24.646853 26802 raft_consensus.cc:374] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:24.646930 26802 raft_consensus.cc:729] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d416b720fa8a48d2a3d9cfb339b2fc68, State: Initialized, Role: FOLLOWER
I20260926 20:51:24.647557 26802 consensus_queue.cc:260] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38395 } }
I20260926 20:51:24.647766 26802 raft_consensus.cc:388] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:24.647868 26802 raft_consensus.cc:482] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:24.647992 26802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:24.650604 26802 raft_consensus.cc:504] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38395 } }
I20260926 20:51:24.651146 26802 leader_election.cc:302] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d416b720fa8a48d2a3d9cfb339b2fc68; no voters:
I20260926 20:51:24.651705 26802 leader_election.cc:288] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:24.651865 26812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:24.652551 26812 raft_consensus.cc:686] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [term 1 LEADER]: Becoming Leader. State: Replica: d416b720fa8a48d2a3d9cfb339b2fc68, State: Running, Role: LEADER
I20260926 20:51:24.653108 26812 consensus_queue.cc:237] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38395 } }
I20260926 20:51:24.653891 26802 sys_catalog.cc:564] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:24.656450 26813 sys_catalog.cc:455] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38395 } } }
I20260926 20:51:24.656311 26814 sys_catalog.cc:455] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d416b720fa8a48d2a3d9cfb339b2fc68. Latest consensus state: current_term: 1 leader_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d416b720fa8a48d2a3d9cfb339b2fc68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38395 } } }
I20260926 20:51:24.656916 26813 sys_catalog.cc:458] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:24.657035 26814 sys_catalog.cc:458] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:24.658383 26820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:24.665197 26820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:24.674690 26820 catalog_manager.cc:1347] Generated new cluster ID: 39f81a7b593a42c78e7a1f3e8b8d6b64
I20260926 20:51:24.674827 26820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:24.690202 26820 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:24.690729 26820 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:24.703487 26820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d416b720fa8a48d2a3d9cfb339b2fc68: Generated new TSK 0
I20260926 20:51:24.704440 26820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:24.710577 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26735
W20260926 20:51:24.712216 26797 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:34535
W20260926 20:51:24.712989 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26735: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:24.713178 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26735
2026-09-26T20:51:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1911 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:24 dist-test-slave-z7j8 krb5kdc[26833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:24 dist-test-slave-z7j8 krb5kdc[26833](info): set up 2 sockets
Sep 26 20:51:24 dist-test-slave-z7j8 krb5kdc[26833](info): commencing operation
krb5kdc: starting...
W20260926 20:51:25.827049 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.007s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:25 dist-test-slave-z7j8 krb5kdc[26833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:26.097967 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:26.436335 26846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:26.436654 26846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:26.436790 26846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:26.444883 26846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:26.445004 26846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:26.445118 26846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:26.445218 26846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:26.445294 26846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:26.455658 26846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34903
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:26.457026 26846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:26.458384 26846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:26.471079 26851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:26.471104 26852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:26.471429 26854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:26.472076 26846 server_base.cc:1034] running on GCE node
I20260926 20:51:26.472856 26846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:26.474293 26846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:26.475530 26846 hybrid_clock.cc:648] HybridClock initialized: now 1790455886475486 us; error 40 us; skew 500 ppm
I20260926 20:51:26.475903 26846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:26 dist-test-slave-z7j8 krb5kdc[26833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:26.479242 26846 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:26.480824 26846 webserver.cc:460] Webserver started at http://127.0.0.1:39755/ using document root <none> and password file <none>
I20260926 20:51:26.481550 26846 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:26.481658 26846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:26.482007 26846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:26.485190 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "281c83ceec6045d8a8e019be72872d49"
format_stamp: "Formatted at 2026-09-26 20:51:26 on dist-test-slave-z7j8"
server_key: "83fa90e88854a5e38b74c8a40a88a9f2"
server_key_iv: "f480d24b9a4316dcbcfd6bac1a638d3a"
server_key_version: "encryptionkey@0"
I20260926 20:51:26.486017 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "281c83ceec6045d8a8e019be72872d49"
format_stamp: "Formatted at 2026-09-26 20:51:26 on dist-test-slave-z7j8"
server_key: "83fa90e88854a5e38b74c8a40a88a9f2"
server_key_iv: "f480d24b9a4316dcbcfd6bac1a638d3a"
server_key_version: "encryptionkey@0"
I20260926 20:51:26.491743 26846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:51:26.495751 26861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:26.497125 26846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:51:26.497327 26846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "281c83ceec6045d8a8e019be72872d49"
format_stamp: "Formatted at 2026-09-26 20:51:26 on dist-test-slave-z7j8"
server_key: "83fa90e88854a5e38b74c8a40a88a9f2"
server_key_iv: "f480d24b9a4316dcbcfd6bac1a638d3a"
server_key_version: "encryptionkey@0"
I20260926 20:51:26.497548 26846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:26.515604 26846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:26.520218 26846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:26.520552 26846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:26.540448 26846 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34903
I20260926 20:51:26.540496 26912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34903 every 8 connection(s)
I20260926 20:51:26.541925 26846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:26.544337 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26846
I20260926 20:51:26.544544 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:26.545092 5432 external_mini_cluster.cc:1428] Setting key a9d0bac2a27e8fc9a15ee28e20a283d8
I20260926 20:51:26.546862 26913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:51:26 dist-test-slave-z7j8 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:51:26.561669 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49: Bootstrap starting.
I20260926 20:51:26.566063 26913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:26.567621 26913 log.cc:824] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:26.571614 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49: No bootstrap required, opened a new log
I20260926 20:51:26.573319 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:26.547093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:52099 (local address 127.0.0.1:34903)
0926 20:51:26.547910 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:26.547926 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:26.547982 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:51:26.550257 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:26.550288 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:26.550943 (+ 655us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:26.551450 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:26.552814 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:26.554960 (+ 2146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:26.556656 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:26.557785 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:26.558019 (+ 234us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:26.563113 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:26.563126 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:26.563148 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:26.563215 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:26.567703 (+ 4488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:26.568969 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:26.568981 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:26.568989 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:26.569277 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:26.569830 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:26.569835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:26.569838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:26.570624 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:26.570961 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:26.571472 (+ 511us) server_negotiation.cc:299] Negotiation successful
0926 20:51:26.571850 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":316,"threads_started":1}
I20260926 20:51:26.577836 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:26.581668 26913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281c83ceec6045d8a8e019be72872d49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260926 20:51:26.582270 26913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:26.582394 26913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 281c83ceec6045d8a8e019be72872d49, State: Initialized, Role: FOLLOWER
I20260926 20:51:26.583112 26913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281c83ceec6045d8a8e019be72872d49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260926 20:51:26.583457 26913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:26.583638 26913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:26.583863 26913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:26.586592 26913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281c83ceec6045d8a8e019be72872d49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260926 20:51:26.587256 26913 leader_election.cc:302] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 281c83ceec6045d8a8e019be72872d49; no voters:
I20260926 20:51:26.588006 26913 leader_election.cc:288] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:26.588150 26923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:26.590281 26913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:26.595252 26923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [term 1 LEADER]: Becoming Leader. State: Replica: 281c83ceec6045d8a8e019be72872d49, State: Running, Role: LEADER
I20260926 20:51:26.595938 26923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281c83ceec6045d8a8e019be72872d49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } }
I20260926 20:51:26.599515 26924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "281c83ceec6045d8a8e019be72872d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281c83ceec6045d8a8e019be72872d49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } } }
I20260926 20:51:26.599411 26925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 281c83ceec6045d8a8e019be72872d49. Latest consensus state: current_term: 1 leader_uuid: "281c83ceec6045d8a8e019be72872d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281c83ceec6045d8a8e019be72872d49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34903 } } }
I20260926 20:51:26.600008 26925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:26.600025 26924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:26.602201 26933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:26.606123 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:26.583596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:36273 (local address 127.0.0.1:34903)
0926 20:51:26.583839 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:26.583850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:26.584436 (+ 586us) server_negotiation.cc:407] Connection header received
0926 20:51:26.584601 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:26.584608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:26.584688 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:26.584820 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:26.586373 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:26.587561 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:26.589887 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:26.590632 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:26.590770 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:26.594903 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:26.594912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:26.594919 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:26.594988 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:26.600039 (+ 5051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:26.601634 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:26.601644 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:26.601651 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:26.602079 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:26.602866 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:26.602871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:26.602874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:26.603273 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:26.605022 (+ 1749us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:26.605263 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 20:51:26.605530 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:51:26.614213 26933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:26.625537 26933 catalog_manager.cc:1347] Generated new cluster ID: 1ed308141e8a4171858f6e7e24010b22
I20260926 20:51:26.625682 26933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:26.641862 26933 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:26.643414 26933 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:26.650331 26933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 281c83ceec6045d8a8e019be72872d49: Generated new TSK 0
I20260926 20:51:26.651500 26933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:26.668188 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26846
I20260926 20:51:26.669667 26908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:26.763820 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26846
W20260926 20:51:26.781271 26831 connection.cc:535] client connection to 127.0.0.1:34903 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34903): Connection reset by peer (error 104)
2026-09-26T20:51:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:26 dist-test-slave-z7j8 krb5kdc[26945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:26 dist-test-slave-z7j8 krb5kdc[26945](info): set up 2 sockets
Sep 26 20:51:26 dist-test-slave-z7j8 krb5kdc[26945](info): commencing operation
krb5kdc: starting...
W20260926 20:51:27.878899 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:28 dist-test-slave-z7j8 krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:28.138137 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:28.474483 26958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:28.474862 26958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:28.475035 26958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:28.482667 26958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:28.482816 26958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:28.482923 26958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:28.483045 26958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:28.483124 26958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:28.493422 26958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42911
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:28.494882 26958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:28.496224 26958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:28.508381 26963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:28.508358 26964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:28.508442 26966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:28.509842 26958 server_base.cc:1034] running on GCE node
I20260926 20:51:28.510831 26958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:28.512501 26958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:28.513793 26958 hybrid_clock.cc:648] HybridClock initialized: now 1790455888513710 us; error 68 us; skew 500 ppm
I20260926 20:51:28.514222 26958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:28 dist-test-slave-z7j8 krb5kdc[26945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:28.517843 26958 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:28.519618 26958 webserver.cc:460] Webserver started at http://127.0.0.1:37029/ using document root <none> and password file <none>
I20260926 20:51:28.520351 26958 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:28.520512 26958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:28.520856 26958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:28.524137 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "ac3b8826db9c434da399f8eb3a8025a0"
format_stamp: "Formatted at 2026-09-26 20:51:28 on dist-test-slave-z7j8"
server_key: "e78613a51813399c998966781f38dc25"
server_key_iv: "20962defc0c6726e675bd0d4b4f6bc39"
server_key_version: "encryptionkey@0"
I20260926 20:51:28.524930 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "ac3b8826db9c434da399f8eb3a8025a0"
format_stamp: "Formatted at 2026-09-26 20:51:28 on dist-test-slave-z7j8"
server_key: "e78613a51813399c998966781f38dc25"
server_key_iv: "20962defc0c6726e675bd0d4b4f6bc39"
server_key_version: "encryptionkey@0"
I20260926 20:51:28.530670 26958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 20:51:28.534808 26973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:28.536398 26958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:28.536638 26958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "ac3b8826db9c434da399f8eb3a8025a0"
format_stamp: "Formatted at 2026-09-26 20:51:28 on dist-test-slave-z7j8"
server_key: "e78613a51813399c998966781f38dc25"
server_key_iv: "20962defc0c6726e675bd0d4b4f6bc39"
server_key_version: "encryptionkey@0"
I20260926 20:51:28.536876 26958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:28.569527 26958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:28.574506 26958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:28.574879 26958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:28.593665 26958 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42911
I20260926 20:51:28.593708 27024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42911 every 8 connection(s)
I20260926 20:51:28.595122 26958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:28.599118 27025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:28.604390 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 26958
I20260926 20:51:28.604573 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:28.609351 5432 external_mini_cluster.cc:1428] Setting key cdac398f323913b6b3a34c523512f60f
I20260926 20:51:28.609930 27025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0: Bootstrap starting.
I20260926 20:51:28.613669 27025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:28.614826 27025 log.cc:824] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:28.617775 27025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0: No bootstrap required, opened a new log
I20260926 20:51:28.623200 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:28.611278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:48651 (local address 127.0.0.1:42911)
0926 20:51:28.612126 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:28.612137 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:28.612177 (+ 40us) server_negotiation.cc:407] Connection header received
0926 20:51:28.613999 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:28.614026 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:28.614512 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:28.614924 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:28.615471 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:28.616805 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:28.618244 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:28.619028 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:28.619244 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:28.620592 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:28.620600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:28.620618 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:28.621235 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:28.621330 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:28.621360 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:28.621555 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:28.621831 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 20:51:28.622015 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":409,"threads_started":1}
I20260926 20:51:28.627130 27025 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3b8826db9c434da399f8eb3a8025a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42911 } }
I20260926 20:51:28.627646 27025 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:28.627674 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:28.627775 27025 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3b8826db9c434da399f8eb3a8025a0, State: Initialized, Role: FOLLOWER
I20260926 20:51:28.628414 27025 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3b8826db9c434da399f8eb3a8025a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42911 } }
I20260926 20:51:28.628685 27025 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:28.628834 27025 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:28.629025 27025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:28.631597 27025 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3b8826db9c434da399f8eb3a8025a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42911 } }
I20260926 20:51:28.632128 27025 leader_election.cc:302] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3b8826db9c434da399f8eb3a8025a0; no voters:
I20260926 20:51:28.632793 27025 leader_election.cc:288] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:28.632957 27034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:28.634996 27025 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:28.637029 27034 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [term 1 LEADER]: Becoming Leader. State: Replica: ac3b8826db9c434da399f8eb3a8025a0, State: Running, Role: LEADER
I20260926 20:51:28.637748 27034 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3b8826db9c434da399f8eb3a8025a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42911 } }
I20260926 20:51:28.647239 27036 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac3b8826db9c434da399f8eb3a8025a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3b8826db9c434da399f8eb3a8025a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42911 } } }
I20260926 20:51:28.647714 27036 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:28.648139 27037 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3b8826db9c434da399f8eb3a8025a0. Latest consensus state: current_term: 1 leader_uuid: "ac3b8826db9c434da399f8eb3a8025a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3b8826db9c434da399f8eb3a8025a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42911 } } }
I20260926 20:51:28.648459 27037 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:28.649479 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:28.632440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:35537 (local address 127.0.0.1:42911)
0926 20:51:28.632670 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:28.632680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:28.640339 (+ 7659us) server_negotiation.cc:407] Connection header received
0926 20:51:28.640577 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:28.640589 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:28.640693 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:28.640847 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:28.642069 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:28.643621 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:28.645605 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:28.646286 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:28.646444 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:28.647872 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:28.647881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:28.647886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:28.647910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:28.647970 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:28.647996 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:28.648131 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:28.648904 (+ 773us) server_negotiation.cc:299] Negotiation successful
0926 20:51:28.648979 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 20:51:28.654273 27052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:51:28.654449 27052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:51:28.654608 27051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:28.656325 27051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:28.664510 27051 catalog_manager.cc:1347] Generated new cluster ID: 34a730c9b8eb4e4c82439f975e52e729
I20260926 20:51:28.664626 27051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:28.674991 27051 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:28.676798 27051 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:28.683655 27051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac3b8826db9c434da399f8eb3a8025a0: Generated new TSK 0
I20260926 20:51:28.684600 27051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:28.689666 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26958
W20260926 20:51:28.690958 27020 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:48651
W20260926 20:51:28.691692 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 26958: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:28.691835 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 26958
W20260926 20:51:28.708482 26943 connection.cc:535] client connection to 127.0.0.1:42911 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42911): Connection reset by peer (error 104)
2026-09-26T20:51:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1927 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:28 dist-test-slave-z7j8 krb5kdc[27057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:28 dist-test-slave-z7j8 krb5kdc[27057](info): set up 2 sockets
Sep 26 20:51:28 dist-test-slave-z7j8 krb5kdc[27057](info): commencing operation
krb5kdc: starting...
W20260926 20:51:29.798883 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:29 dist-test-slave-z7j8 krb5kdc[27057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:51:30.061692 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:30.395989 27070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:30.396289 27070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:30.396502 27070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:30.403745 27070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:30.403860 27070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:30.403985 27070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:30.404068 27070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:30.404150 27070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:30.417011 27070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:30.418540 27070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:30.420274 27070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:30.432741 27075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:30.432860 27076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:30.432971 27078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:30.433658 27070 server_base.cc:1034] running on GCE node
I20260926 20:51:30.434473 27070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:30.435985 27070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:30.437237 27070 hybrid_clock.cc:648] HybridClock initialized: now 1790455890437177 us; error 50 us; skew 500 ppm
I20260926 20:51:30.437620 27070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:30 dist-test-slave-z7j8 krb5kdc[27057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:30.445150 27070 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:51:30.446973 27070 webserver.cc:460] Webserver started at http://127.0.0.1:33315/ using document root <none> and password file <none>
I20260926 20:51:30.447712 27070 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:30.447877 27070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:30.448202 27070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:30.451365 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "988c5cb7630e4a7f85a9989d7d50519a"
format_stamp: "Formatted at 2026-09-26 20:51:30 on dist-test-slave-z7j8"
server_key: "d4c15db1c6c0c21192c0643394de7b7b"
server_key_iv: "1bf8ce3c286f034512f74c5942c77263"
server_key_version: "encryptionkey@0"
I20260926 20:51:30.452144 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "988c5cb7630e4a7f85a9989d7d50519a"
format_stamp: "Formatted at 2026-09-26 20:51:30 on dist-test-slave-z7j8"
server_key: "d4c15db1c6c0c21192c0643394de7b7b"
server_key_iv: "1bf8ce3c286f034512f74c5942c77263"
server_key_version: "encryptionkey@0"
I20260926 20:51:30.457652 27070 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:51:30.461524 27085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:30.463183 27070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:30.463407 27070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "988c5cb7630e4a7f85a9989d7d50519a"
format_stamp: "Formatted at 2026-09-26 20:51:30 on dist-test-slave-z7j8"
server_key: "d4c15db1c6c0c21192c0643394de7b7b"
server_key_iv: "1bf8ce3c286f034512f74c5942c77263"
server_key_version: "encryptionkey@0"
I20260926 20:51:30.463649 27070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:30.469795 27070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:30.473860 27070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:30.474145 27070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:30.493075 27070 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40701
I20260926 20:51:30.493122 27136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40701 every 8 connection(s)
I20260926 20:51:30.494679 27070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:30.497334 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27070
I20260926 20:51:30.497517 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:30.498059 5432 external_mini_cluster.cc:1428] Setting key feeb779beceae83bb8ea4e19bef45151
I20260926 20:51:30.499130 27137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:30.505861 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:30.499919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:43009 (local address 127.0.0.1:40701)
0926 20:51:30.500714 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:30.500727 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:30.500803 (+ 76us) server_negotiation.cc:407] Connection header received
0926 20:51:30.502358 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:30.502378 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:30.502807 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:30.503228 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:30.504269 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:30.504275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:30.504288 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:30.504302 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:30.504371 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:30.504391 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:30.504629 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:30.504783 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 20:51:30.504967 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":343,"threads_started":1}
I20260926 20:51:30.509598 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:30.511729 27137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a: Bootstrap starting.
I20260926 20:51:30.515013 27137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:30.516343 27137 log.cc:824] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:30.516913 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:30.513832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:37255 (local address 127.0.0.1:40701)
0926 20:51:30.514013 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:30.514019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:30.514501 (+ 482us) server_negotiation.cc:407] Connection header received
0926 20:51:30.514691 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:30.514704 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:30.514761 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:30.514861 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:30.515857 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:30.515863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:30.515867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:30.515891 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:30.515948 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:30.515971 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:30.516082 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:30.516262 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 20:51:30.516544 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:51:30.519409 27137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a: No bootstrap required, opened a new log
I20260926 20:51:30.527161 27137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988c5cb7630e4a7f85a9989d7d50519a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260926 20:51:30.527520 27137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:30.527593 27137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988c5cb7630e4a7f85a9989d7d50519a, State: Initialized, Role: FOLLOWER
I20260926 20:51:30.528076 27137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988c5cb7630e4a7f85a9989d7d50519a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260926 20:51:30.528270 27137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:30.528373 27137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:30.528497 27137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:30.530937 27137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988c5cb7630e4a7f85a9989d7d50519a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260926 20:51:30.531436 27137 leader_election.cc:302] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 988c5cb7630e4a7f85a9989d7d50519a; no voters:
I20260926 20:51:30.531955 27137 leader_election.cc:288] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:30.532119 27147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:30.532802 27147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [term 1 LEADER]: Becoming Leader. State: Replica: 988c5cb7630e4a7f85a9989d7d50519a, State: Running, Role: LEADER
I20260926 20:51:30.533350 27147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988c5cb7630e4a7f85a9989d7d50519a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } }
I20260926 20:51:30.534025 27137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:30.536314 27148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "988c5cb7630e4a7f85a9989d7d50519a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988c5cb7630e4a7f85a9989d7d50519a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } } }
I20260926 20:51:30.536767 27148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:30.536623 27149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988c5cb7630e4a7f85a9989d7d50519a. Latest consensus state: current_term: 1 leader_uuid: "988c5cb7630e4a7f85a9989d7d50519a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988c5cb7630e4a7f85a9989d7d50519a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40701 } } }
I20260926 20:51:30.536921 27149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:30.537672 27156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:30.544309 27156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:30.556548 27156 catalog_manager.cc:1347] Generated new cluster ID: a39bfe653c564decb65c48009e359555
I20260926 20:51:30.556689 27156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:30.569924 27156 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:30.570485 27156 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:30.584846 27156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 988c5cb7630e4a7f85a9989d7d50519a: Generated new TSK 0
I20260926 20:51:30.585866 27156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:30.595224 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27070
W20260926 20:51:30.596696 27132 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:43009
W20260926 20:51:30.597502 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27070: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:30.597704 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27070
2026-09-26T20:51:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1906 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:30 dist-test-slave-z7j8 krb5kdc[27168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:30 dist-test-slave-z7j8 krb5kdc[27168](info): set up 2 sockets
Sep 26 20:51:30 dist-test-slave-z7j8 krb5kdc[27168](info): commencing operation
krb5kdc: starting...
W20260926 20:51:31.707008 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.008s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:31 dist-test-slave-z7j8 krb5kdc[27168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:31.974449 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44751
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:32.314543 27181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:32.314910 27181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:32.315081 27181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:32.322656 27181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:32.322788 27181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:32.322882 27181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:32.323000 27181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:32.323073 27181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:32.334862 27181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:32.336323 27181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:32.337805 27181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:32.350262 27187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:32.350181 27186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:32.351477 27189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:32.352304 27181 server_base.cc:1034] running on GCE node
I20260926 20:51:32.353137 27181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:32.354807 27181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:32.356050 27181 hybrid_clock.cc:648] HybridClock initialized: now 1790455892355974 us; error 82 us; skew 500 ppm
I20260926 20:51:32.356474 27181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:32 dist-test-slave-z7j8 krb5kdc[27168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455892, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:32.360247 27181 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:32.361837 27181 webserver.cc:460] Webserver started at http://127.5.78.62:33487/ using document root <none> and password file <none>
I20260926 20:51:32.362685 27181 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:32.362841 27181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:32.363181 27181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:32.366359 27181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "c630b17f73f1472daa6ff040e95382ec"
format_stamp: "Formatted at 2026-09-26 20:51:32 on dist-test-slave-z7j8"
server_key: "eb2af2bbc236a4a0677b37159a32d52e"
server_key_iv: "40fe3a9ca87e0236f50d94fb5771c760"
server_key_version: "encryptionkey@0"
I20260926 20:51:32.367261 27181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "c630b17f73f1472daa6ff040e95382ec"
format_stamp: "Formatted at 2026-09-26 20:51:32 on dist-test-slave-z7j8"
server_key: "eb2af2bbc236a4a0677b37159a32d52e"
server_key_iv: "40fe3a9ca87e0236f50d94fb5771c760"
server_key_version: "encryptionkey@0"
I20260926 20:51:32.373224 27181 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260926 20:51:32.377365 27196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:32.379135 27181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 20:51:32.379377 27181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "c630b17f73f1472daa6ff040e95382ec"
format_stamp: "Formatted at 2026-09-26 20:51:32 on dist-test-slave-z7j8"
server_key: "eb2af2bbc236a4a0677b37159a32d52e"
server_key_iv: "40fe3a9ca87e0236f50d94fb5771c760"
server_key_version: "encryptionkey@0"
I20260926 20:51:32.379626 27181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:32.397985 27181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:32.402559 27181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:32.402875 27181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:32.423069 27181 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:44751
I20260926 20:51:32.423082 27247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:44751 every 8 connection(s)
I20260926 20:51:32.424674 27181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:32.428686 27248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:32.431031 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27181
I20260926 20:51:32.431278 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:32.431743 5432 external_mini_cluster.cc:1428] Setting key c100d891e81c8e8a4d511d3fb018ff04
I20260926 20:51:32.444346 27248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec: Bootstrap starting.
Sep 26 20:51:32 dist-test-slave-z7j8 krb5kdc[27168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:32.448699 27248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:32.449966 27248 log.cc:824] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:32.453271 27248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec: No bootstrap required, opened a new log
I20260926 20:51:32.458690 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:32.433450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39587 (local address 127.5.78.62:44751)
0926 20:51:32.434453 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:32.434478 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:32.434528 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:51:32.436229 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:32.436255 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:32.436720 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:32.437065 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:32.438950 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:32.440388 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:32.442550 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:32.443365 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:32.443547 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:32.449049 (+ 5502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:32.449059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:32.449079 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:32.449134 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:32.452757 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:32.453800 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:32.453808 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:32.453815 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:32.454082 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:32.454822 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:32.454827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:32.454830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:32.455660 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:32.456043 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:32.456640 (+ 597us) server_negotiation.cc:299] Negotiation successful
0926 20:51:32.457112 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":467,"threads_started":1}
I20260926 20:51:32.464257 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:32.464308 27248 raft_consensus.cc:348] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c630b17f73f1472daa6ff040e95382ec" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44751 } }
I20260926 20:51:32.464815 27248 raft_consensus.cc:374] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:32.464926 27248 raft_consensus.cc:729] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c630b17f73f1472daa6ff040e95382ec, State: Initialized, Role: FOLLOWER
I20260926 20:51:32.465665 27248 consensus_queue.cc:260] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c630b17f73f1472daa6ff040e95382ec" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44751 } }
I20260926 20:51:32.465960 27248 raft_consensus.cc:388] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:32.466104 27248 raft_consensus.cc:482] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:32.466274 27248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:32.469857 27248 raft_consensus.cc:504] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c630b17f73f1472daa6ff040e95382ec" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44751 } }
I20260926 20:51:32.470639 27248 leader_election.cc:302] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c630b17f73f1472daa6ff040e95382ec; no voters:
I20260926 20:51:32.471387 27248 leader_election.cc:288] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:32.471611 27258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:32.473018 27258 raft_consensus.cc:686] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [term 1 LEADER]: Becoming Leader. State: Replica: c630b17f73f1472daa6ff040e95382ec, State: Running, Role: LEADER
I20260926 20:51:32.473665 27258 consensus_queue.cc:237] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c630b17f73f1472daa6ff040e95382ec" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44751 } }
I20260926 20:51:32.474077 27248 sys_catalog.cc:564] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:32.476208 27260 sys_catalog.cc:455] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader c630b17f73f1472daa6ff040e95382ec. Latest consensus state: current_term: 1 leader_uuid: "c630b17f73f1472daa6ff040e95382ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c630b17f73f1472daa6ff040e95382ec" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44751 } } }
I20260926 20:51:32.476598 27260 sys_catalog.cc:458] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:32.477262 27259 sys_catalog.cc:455] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c630b17f73f1472daa6ff040e95382ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c630b17f73f1472daa6ff040e95382ec" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44751 } } }
I20260926 20:51:32.477630 27259 sys_catalog.cc:458] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:32.478469 27263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:32.492063 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:32.469714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44463 (local address 127.5.78.62:44751)
0926 20:51:32.469976 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:32.469982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:32.470494 (+ 512us) server_negotiation.cc:407] Connection header received
0926 20:51:32.470602 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:32.470605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:32.470676 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:32.470761 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:32.472902 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:32.474160 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:32.476433 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:32.477252 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:32.477452 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:32.481177 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:32.481188 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:32.481196 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:32.481255 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:32.486313 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:32.487745 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:32.487758 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:32.487764 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:32.487987 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:32.488732 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:32.488740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:32.488746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:32.489309 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:32.489684 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:32.491195 (+ 1511us) server_negotiation.cc:299] Negotiation successful
0926 20:51:32.491476 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:51:32.497648 27263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:32.507848 27263 catalog_manager.cc:1347] Generated new cluster ID: 5a067beaa75e4656a1de1122274b077e
I20260926 20:51:32.508044 27263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:32.530741 27263 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:32.532790 27263 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:32.540688 27263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c630b17f73f1472daa6ff040e95382ec: Generated new TSK 0
I20260926 20:51:32.541874 27263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:32.548967 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27181
I20260926 20:51:32.550438 27243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:32.642549 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27181
W20260926 20:51:32.659806 27166 connection.cc:535] client connection to 127.5.78.62:44751 recv error: Network error: failed to read from TLS socket (remote: 127.5.78.62:44751): Connection reset by peer (error 104)
2026-09-26T20:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:32 dist-test-slave-z7j8 krb5kdc[27280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:32 dist-test-slave-z7j8 krb5kdc[27280](info): set up 2 sockets
Sep 26 20:51:32 dist-test-slave-z7j8 krb5kdc[27280](info): commencing operation
krb5kdc: starting...
W20260926 20:51:33.754750 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:33 dist-test-slave-z7j8 krb5kdc[27280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:34.018092 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38689
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:38689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:34.355271 27293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:34.355590 27293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:34.355770 27293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:34.363175 27293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:34.363299 27293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:34.363396 27293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:34.363507 27293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:34.363587 27293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:34.373757 27293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:38689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:38689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:34.375236 27293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:34.376699 27293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:34.389599 27301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:34.389664 27298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:34.389642 27299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:34.390426 27293 server_base.cc:1034] running on GCE node
I20260926 20:51:34.391253 27293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:34.392778 27293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:34.394028 27293 hybrid_clock.cc:648] HybridClock initialized: now 1790455894393987 us; error 40 us; skew 500 ppm
I20260926 20:51:34.394441 27293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:34 dist-test-slave-z7j8 krb5kdc[27280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:34.397650 27293 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:34.399274 27293 webserver.cc:460] Webserver started at http://127.5.78.62:42691/ using document root <none> and password file <none>
I20260926 20:51:34.399960 27293 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:34.400063 27293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:34.400308 27293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:34.403512 27293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "cf3ab9f841be45e8b12b8b90619f7f2f"
format_stamp: "Formatted at 2026-09-26 20:51:34 on dist-test-slave-z7j8"
server_key: "29f9879a1d4049358622c8a6202073f4"
server_key_iv: "1f749daca072d5f49b6737c63c94b45c"
server_key_version: "encryptionkey@0"
I20260926 20:51:34.404289 27293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "cf3ab9f841be45e8b12b8b90619f7f2f"
format_stamp: "Formatted at 2026-09-26 20:51:34 on dist-test-slave-z7j8"
server_key: "29f9879a1d4049358622c8a6202073f4"
server_key_iv: "1f749daca072d5f49b6737c63c94b45c"
server_key_version: "encryptionkey@0"
I20260926 20:51:34.409901 27293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:51:34.413786 27308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:34.415279 27293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:34.415436 27293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "cf3ab9f841be45e8b12b8b90619f7f2f"
format_stamp: "Formatted at 2026-09-26 20:51:34 on dist-test-slave-z7j8"
server_key: "29f9879a1d4049358622c8a6202073f4"
server_key_iv: "1f749daca072d5f49b6737c63c94b45c"
server_key_version: "encryptionkey@0"
I20260926 20:51:34.415608 27293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:34.428488 27293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:34.432953 27293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:34.433211 27293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:34.452694 27293 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:38689
I20260926 20:51:34.452713 27359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:38689 every 8 connection(s)
I20260926 20:51:34.454231 27293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:34.458323 27360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:34.464496 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27293
I20260926 20:51:34.464696 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:34.465278 5432 external_mini_cluster.cc:1428] Setting key 03d3adb0376a631fac08e28c0a0a59de
I20260926 20:51:34.469866 27360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f: Bootstrap starting.
I20260926 20:51:34.473418 27360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:34.474737 27360 log.cc:824] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:34.477778 27360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f: No bootstrap required, opened a new log
I20260926 20:51:34.479655 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:34.467359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44705 (local address 127.5.78.62:38689)
0926 20:51:34.468283 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:34.468297 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:34.468344 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:51:34.470246 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:34.470272 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:34.470775 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:34.471146 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:34.471834 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:34.473185 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:34.475132 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:34.475863 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:34.476075 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:34.477368 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:34.477375 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:34.477390 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:34.477977 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:34.478071 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:34.478097 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:34.478280 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:34.478553 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 20:51:34.478709 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":449,"threads_started":1}
I20260926 20:51:34.483680 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:34.486645 27360 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38689 } }
I20260926 20:51:34.487072 27360 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:34.487192 27360 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf3ab9f841be45e8b12b8b90619f7f2f, State: Initialized, Role: FOLLOWER
I20260926 20:51:34.487834 27360 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38689 } }
I20260926 20:51:34.488163 27360 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:34.488380 27360 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:34.488574 27360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:34.491686 27360 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38689 } }
I20260926 20:51:34.492268 27360 leader_election.cc:302] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf3ab9f841be45e8b12b8b90619f7f2f; no voters:
I20260926 20:51:34.492962 27360 leader_election.cc:288] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:34.493081 27370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:34.493995 27370 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [term 1 LEADER]: Becoming Leader. State: Replica: cf3ab9f841be45e8b12b8b90619f7f2f, State: Running, Role: LEADER
I20260926 20:51:34.494712 27370 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38689 } }
I20260926 20:51:34.495492 27360 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:34.495515 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:34.488189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54309 (local address 127.5.78.62:38689)
0926 20:51:34.488397 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:34.488405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:34.488853 (+ 448us) server_negotiation.cc:407] Connection header received
0926 20:51:34.489006 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:34.489014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:34.489095 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:34.489235 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:34.489677 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:34.490837 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:34.492215 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:34.492959 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:34.493096 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:34.494378 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:34.494384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:34.494388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:34.494438 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:34.494519 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:34.494574 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:34.494712 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:34.495123 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 20:51:34.495188 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:51:34.498225 27371 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38689 } } }
I20260926 20:51:34.498340 27372 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf3ab9f841be45e8b12b8b90619f7f2f. Latest consensus state: current_term: 1 leader_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3ab9f841be45e8b12b8b90619f7f2f" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 38689 } } }
I20260926 20:51:34.498700 27371 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:34.498781 27372 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:34.504485 27376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:34.510064 27376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:34.519163 27376 catalog_manager.cc:1347] Generated new cluster ID: 2bc3a60ec42d41b980f1cd318e3a59fe
I20260926 20:51:34.519325 27376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:34.542852 27376 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:34.544955 27376 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:34.554603 27376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf3ab9f841be45e8b12b8b90619f7f2f: Generated new TSK 0
I20260926 20:51:34.555637 27376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:34.567461 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27293
W20260926 20:51:34.568684 27355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44705
W20260926 20:51:34.569585 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27293: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:34.569739 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27293
W20260926 20:51:34.584566 27278 connection.cc:535] client connection to 127.5.78.62:38689 recv error: Network error: failed to read from TLS socket (remote: 127.5.78.62:38689): Connection reset by peer (error 104)
2026-09-26T20:51:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1924 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:34 dist-test-slave-z7j8 krb5kdc[27391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:34 dist-test-slave-z7j8 krb5kdc[27391](info): set up 2 sockets
Sep 26 20:51:34 dist-test-slave-z7j8 krb5kdc[27391](info): commencing operation
krb5kdc: starting...
W20260926 20:51:35.682858 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:35 dist-test-slave-z7j8 krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:35.950814 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46255
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:36.292780 27404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:36.293105 27404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:36.293200 27404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:36.300688 27404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:36.300822 27404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:36.300871 27404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:36.300930 27404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:36.300976 27404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:36.311198 27404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:46255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:36.312510 27404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:36.313827 27404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:36.326205 27409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:36.326221 27412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:36.326195 27410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:36.327236 27404 server_base.cc:1034] running on GCE node
I20260926 20:51:36.328104 27404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:36.329662 27404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:36.330926 27404 hybrid_clock.cc:648] HybridClock initialized: now 1790455896330864 us; error 47 us; skew 500 ppm
I20260926 20:51:36.331321 27404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:36 dist-test-slave-z7j8 krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455896, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:36.341387 27404 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:36.343114 27404 webserver.cc:460] Webserver started at http://127.5.78.62:41119/ using document root <none> and password file <none>
I20260926 20:51:36.343799 27404 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:36.343915 27404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:36.344185 27404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:36.347414 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "9fdc8466b1c14ec58d8edad150d41200"
format_stamp: "Formatted at 2026-09-26 20:51:36 on dist-test-slave-z7j8"
server_key: "cef6633c0afc9cd1e8d2662ddcdfe59a"
server_key_iv: "8f1b2fd5ce84b897a1bb59a18ca14fbe"
server_key_version: "encryptionkey@0"
I20260926 20:51:36.348361 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "9fdc8466b1c14ec58d8edad150d41200"
format_stamp: "Formatted at 2026-09-26 20:51:36 on dist-test-slave-z7j8"
server_key: "cef6633c0afc9cd1e8d2662ddcdfe59a"
server_key_iv: "8f1b2fd5ce84b897a1bb59a18ca14fbe"
server_key_version: "encryptionkey@0"
I20260926 20:51:36.355026 27404 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 20:51:36.359007 27419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:36.360575 27404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:51:36.360788 27404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "9fdc8466b1c14ec58d8edad150d41200"
format_stamp: "Formatted at 2026-09-26 20:51:36 on dist-test-slave-z7j8"
server_key: "cef6633c0afc9cd1e8d2662ddcdfe59a"
server_key_iv: "8f1b2fd5ce84b897a1bb59a18ca14fbe"
server_key_version: "encryptionkey@0"
I20260926 20:51:36.361047 27404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:36.367348 27404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:36.371881 27404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:36.372212 27404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:36.392047 27404 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:46255
I20260926 20:51:36.392084 27470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:46255 every 8 connection(s)
I20260926 20:51:36.393532 27404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:36.397004 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27404
I20260926 20:51:36.397199 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:36.397660 5432 external_mini_cluster.cc:1428] Setting key e4dc491620d6b6fbc2f84c07f6f5cfb0
I20260926 20:51:36.398015 27471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:36.406716 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:36.399676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53553 (local address 127.5.78.62:46255)
0926 20:51:36.400490 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:36.400503 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:36.400545 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:51:36.403182 (+ 2637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:36.403206 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:36.403563 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:36.403905 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:36.405068 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:36.405075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:36.405090 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:36.405104 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:36.405165 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:36.405191 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:36.405369 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:36.405469 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 20:51:36.405670 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":367,"threads_started":1}
I20260926 20:51:36.410719 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:36.413576 27471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200: Bootstrap starting.
I20260926 20:51:36.417697 27471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:36.419147 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:36.416309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52747 (local address 127.5.78.62:46255)
0926 20:51:36.416519 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:36.416525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:36.416930 (+ 405us) server_negotiation.cc:407] Connection header received
0926 20:51:36.417096 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:36.417101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:36.417164 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:36.417277 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:36.418269 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:36.418273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:36.418277 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:36.418292 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:36.418332 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:36.418347 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:36.418561 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:36.418734 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 20:51:36.418788 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:51:36.419189 27471 log.cc:824] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:36.422700 27471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200: No bootstrap required, opened a new log
I20260926 20:51:36.430836 27471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc8466b1c14ec58d8edad150d41200" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46255 } }
I20260926 20:51:36.431293 27471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:36.431404 27471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fdc8466b1c14ec58d8edad150d41200, State: Initialized, Role: FOLLOWER
I20260926 20:51:36.432111 27471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc8466b1c14ec58d8edad150d41200" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46255 } }
I20260926 20:51:36.432337 27471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:36.432507 27471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:36.432675 27471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:36.435312 27471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc8466b1c14ec58d8edad150d41200" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46255 } }
I20260926 20:51:36.435879 27471 leader_election.cc:302] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fdc8466b1c14ec58d8edad150d41200; no voters:
I20260926 20:51:36.436534 27471 leader_election.cc:288] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:36.436690 27481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:36.437371 27481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [term 1 LEADER]: Becoming Leader. State: Replica: 9fdc8466b1c14ec58d8edad150d41200, State: Running, Role: LEADER
I20260926 20:51:36.437913 27481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc8466b1c14ec58d8edad150d41200" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46255 } }
I20260926 20:51:36.438779 27471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:36.440872 27483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fdc8466b1c14ec58d8edad150d41200. Latest consensus state: current_term: 1 leader_uuid: "9fdc8466b1c14ec58d8edad150d41200" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc8466b1c14ec58d8edad150d41200" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46255 } } }
I20260926 20:51:36.441250 27483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:36.442173 27487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:36.442003 27482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fdc8466b1c14ec58d8edad150d41200" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc8466b1c14ec58d8edad150d41200" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 46255 } } }
I20260926 20:51:36.443297 27482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:36.448155 27487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:36.457778 27487 catalog_manager.cc:1347] Generated new cluster ID: 9accc076514145bdb97ba9925e69fe62
I20260926 20:51:36.457921 27487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:36.480134 27487 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:36.480598 27487 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:36.487064 27487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fdc8466b1c14ec58d8edad150d41200: Generated new TSK 0
I20260926 20:51:36.487973 27487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:36.503432 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27404
W20260926 20:51:36.505010 27466 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53553
W20260926 20:51:36.505823 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27404: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:36.506003 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27404
2026-09-26T20:51:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1938 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:36 dist-test-slave-z7j8 krb5kdc[27502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:36 dist-test-slave-z7j8 krb5kdc[27502](info): set up 2 sockets
Sep 26 20:51:36 dist-test-slave-z7j8 krb5kdc[27502](info): commencing operation
krb5kdc: starting...
W20260926 20:51:37.626917 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:37 dist-test-slave-z7j8 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:37.894837 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:36169
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:38.233246 27515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:38.233634 27515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:38.233820 27515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:38.241230 27515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:38.241411 27515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:38.241528 27515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:38.241648 27515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:38.241729 27515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:38.252197 27515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:36169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:38.253700 27515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:38.255142 27515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:38.267342 27521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:38.267288 27520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:38.267390 27523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:38.268123 27515 server_base.cc:1034] running on GCE node
I20260926 20:51:38.268903 27515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:38.270542 27515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:38.271811 27515 hybrid_clock.cc:648] HybridClock initialized: now 1790455898271738 us; error 68 us; skew 500 ppm
I20260926 20:51:38.272221 27515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:38 dist-test-slave-z7j8 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455898, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:38.275704 27515 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:38.277293 27515 webserver.cc:460] Webserver started at http://127.5.78.62:42075/ using document root <none> and password file <none>
I20260926 20:51:38.278024 27515 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:38.278174 27515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:38.278554 27515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:38.281687 27515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "bf82d9682656497db093b90b15e5174c"
format_stamp: "Formatted at 2026-09-26 20:51:38 on dist-test-slave-z7j8"
server_key: "81861465eccd4a6d9d505accdc318c6c"
server_key_iv: "731cc4ac8959b3c5cd872d88ca2154d0"
server_key_version: "encryptionkey@0"
I20260926 20:51:38.282516 27515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "bf82d9682656497db093b90b15e5174c"
format_stamp: "Formatted at 2026-09-26 20:51:38 on dist-test-slave-z7j8"
server_key: "81861465eccd4a6d9d505accdc318c6c"
server_key_iv: "731cc4ac8959b3c5cd872d88ca2154d0"
server_key_version: "encryptionkey@0"
I20260926 20:51:38.287690 27515 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:51:38.291374 27530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:38.292699 27515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:38.292912 27515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "bf82d9682656497db093b90b15e5174c"
format_stamp: "Formatted at 2026-09-26 20:51:38 on dist-test-slave-z7j8"
server_key: "81861465eccd4a6d9d505accdc318c6c"
server_key_iv: "731cc4ac8959b3c5cd872d88ca2154d0"
server_key_version: "encryptionkey@0"
I20260926 20:51:38.293119 27515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:38.312829 27515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:38.317390 27515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:38.317700 27515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:38.337419 27515 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:36169
I20260926 20:51:38.337455 27581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:36169 every 8 connection(s)
I20260926 20:51:38.338909 27515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:38.341064 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27515
I20260926 20:51:38.341280 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:38.341755 5432 external_mini_cluster.cc:1428] Setting key abac3e4fc6e76047b77a70e6f61ba646
I20260926 20:51:38.343405 27582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:51:38 dist-test-slave-z7j8 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:38.358039 27582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c: Bootstrap starting.
I20260926 20:51:38.361526 27582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:38.362797 27582 log.cc:824] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:38.365901 27582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c: No bootstrap required, opened a new log
I20260926 20:51:38.366760 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:38.343659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54899 (local address 127.5.78.62:36169)
0926 20:51:38.344430 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:38.344445 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:38.344495 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:51:38.346648 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:38.346676 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:38.347201 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:38.347670 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:38.349011 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:38.351171 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:38.352499 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:38.353591 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:38.353800 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:38.359034 (+ 5234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:38.359044 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:38.359060 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:38.359104 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:38.362329 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:38.363255 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:38.363261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:38.363267 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:38.363441 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:38.363857 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:38.363863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:38.363866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:38.364439 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:38.364732 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:38.365198 (+ 466us) server_negotiation.cc:299] Negotiation successful
0926 20:51:38.365651 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":326,"threads_started":1}
I20260926 20:51:38.370527 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:38.374158 27582 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf82d9682656497db093b90b15e5174c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 36169 } }
I20260926 20:51:38.374815 27582 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:38.374943 27582 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf82d9682656497db093b90b15e5174c, State: Initialized, Role: FOLLOWER
I20260926 20:51:38.375671 27582 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf82d9682656497db093b90b15e5174c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 36169 } }
I20260926 20:51:38.375980 27582 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:38.376155 27582 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:38.376340 27582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:38.378669 27582 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf82d9682656497db093b90b15e5174c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 36169 } }
I20260926 20:51:38.379174 27582 leader_election.cc:302] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf82d9682656497db093b90b15e5174c; no voters:
I20260926 20:51:38.379863 27582 leader_election.cc:288] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:38.379971 27592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:38.381337 27592 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [term 1 LEADER]: Becoming Leader. State: Replica: bf82d9682656497db093b90b15e5174c, State: Running, Role: LEADER
I20260926 20:51:38.381923 27592 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf82d9682656497db093b90b15e5174c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 36169 } }
I20260926 20:51:38.382326 27582 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:38.388491 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf82d9682656497db093b90b15e5174c. Latest consensus state: current_term: 1 leader_uuid: "bf82d9682656497db093b90b15e5174c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf82d9682656497db093b90b15e5174c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 36169 } } }
I20260926 20:51:38.389030 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:38.389791 27593 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf82d9682656497db093b90b15e5174c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf82d9682656497db093b90b15e5174c" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 36169 } } }
I20260926 20:51:38.390142 27593 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:38.392109 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:38.374696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51469 (local address 127.5.78.62:36169)
0926 20:51:38.374909 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:38.374918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:38.375629 (+ 711us) server_negotiation.cc:407] Connection header received
0926 20:51:38.375798 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:38.375802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:38.375863 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:38.375976 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:38.377499 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:38.378246 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:38.379641 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:38.380284 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:38.380432 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:38.383341 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:38.383347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:38.383351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:38.383387 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:38.386863 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:38.387846 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:38.387854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:38.387857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:38.388028 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:38.388533 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:38.388538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:38.388541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:38.389064 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:38.390976 (+ 1912us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:38.391183 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 20:51:38.391502 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:51:38.399516 27601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:38.405387 27601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:38.415153 27601 catalog_manager.cc:1347] Generated new cluster ID: 1281fd7c78d24bdd99326e76ccbfde54
I20260926 20:51:38.415349 27601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:38.433640 27601 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:38.435237 27601 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:38.443782 27601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf82d9682656497db093b90b15e5174c: Generated new TSK 0
I20260926 20:51:38.444658 27601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:38.451500 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27515
I20260926 20:51:38.452788 27577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:38.545575 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27515
2026-09-26T20:51:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2037 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:38 dist-test-slave-z7j8 krb5kdc[27615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:38 dist-test-slave-z7j8 krb5kdc[27615](info): set up 2 sockets
Sep 26 20:51:38 dist-test-slave-z7j8 krb5kdc[27615](info): commencing operation
krb5kdc: starting...
W20260926 20:51:39.654953 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.008s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:39 dist-test-slave-z7j8 krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:39.918803 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:37721
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:37721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:40.256291 27628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:40.256585 27628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:40.256757 27628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:40.264231 27628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:40.264358 27628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:40.264410 27628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:40.264539 27628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:40.264639 27628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:40.275475 27628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:37721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:37721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:40.276872 27628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:40.278301 27628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:40.290550 27634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:40.290596 27636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:40.290627 27633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:40.291767 27628 server_base.cc:1034] running on GCE node
I20260926 20:51:40.292595 27628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:40.294026 27628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:40.295233 27628 hybrid_clock.cc:648] HybridClock initialized: now 1790455900295188 us; error 46 us; skew 500 ppm
I20260926 20:51:40.295573 27628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:40 dist-test-slave-z7j8 krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455900, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:40.298818 27628 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:40.300446 27628 webserver.cc:460] Webserver started at http://127.5.78.62:41197/ using document root <none> and password file <none>
I20260926 20:51:40.301186 27628 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:40.301316 27628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:40.301638 27628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:40.304749 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "8fcb07d7a2554e4c88001e5e8899b6a0"
format_stamp: "Formatted at 2026-09-26 20:51:40 on dist-test-slave-z7j8"
server_key: "09d845ba309d8e837cde6b7a43b58a84"
server_key_iv: "0157d633d5c1e8ba84fabec5e526612c"
server_key_version: "encryptionkey@0"
I20260926 20:51:40.305529 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "8fcb07d7a2554e4c88001e5e8899b6a0"
format_stamp: "Formatted at 2026-09-26 20:51:40 on dist-test-slave-z7j8"
server_key: "09d845ba309d8e837cde6b7a43b58a84"
server_key_iv: "0157d633d5c1e8ba84fabec5e526612c"
server_key_version: "encryptionkey@0"
I20260926 20:51:40.310820 27628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:51:40.314337 27643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:40.315768 27628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:40.315985 27628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "8fcb07d7a2554e4c88001e5e8899b6a0"
format_stamp: "Formatted at 2026-09-26 20:51:40 on dist-test-slave-z7j8"
server_key: "09d845ba309d8e837cde6b7a43b58a84"
server_key_iv: "0157d633d5c1e8ba84fabec5e526612c"
server_key_version: "encryptionkey@0"
I20260926 20:51:40.316167 27628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:40.333473 27628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:40.337949 27628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:40.338280 27628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:40.357465 27628 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:37721
I20260926 20:51:40.357504 27694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:37721 every 8 connection(s)
I20260926 20:51:40.359093 27628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:40.363189 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:40.365094 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27628
I20260926 20:51:40.365273 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:40.365801 5432 external_mini_cluster.cc:1428] Setting key 23f26f901ab7a4a956f44150699fa0ae
I20260926 20:51:40.378046 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0: Bootstrap starting.
I20260926 20:51:40.382131 27695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:40.383450 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:40.368206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47653 (local address 127.5.78.62:37721)
0926 20:51:40.369311 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:40.369332 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:40.369770 (+ 438us) server_negotiation.cc:407] Connection header received
0926 20:51:40.372278 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:40.372310 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:40.372902 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:40.373419 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:40.373994 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:40.376149 (+ 2155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:40.377594 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:40.378731 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:40.378913 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:40.380232 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:40.380244 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:40.380263 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:40.381009 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:40.381124 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:40.381158 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:40.381277 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:40.381672 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 20:51:40.381968 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":496,"threads_started":1}
I20260926 20:51:40.383598 27695 log.cc:824] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:40.387137 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0: No bootstrap required, opened a new log
I20260926 20:51:40.387910 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:40.395884 27695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37721 } }
I20260926 20:51:40.396317 27695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:40.396436 27695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fcb07d7a2554e4c88001e5e8899b6a0, State: Initialized, Role: FOLLOWER
I20260926 20:51:40.397141 27695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37721 } }
I20260926 20:51:40.397440 27695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:40.397660 27695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:40.397848 27695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:40.399574 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:40.392440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33045 (local address 127.5.78.62:37721)
0926 20:51:40.392632 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:40.392639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:40.393071 (+ 432us) server_negotiation.cc:407] Connection header received
0926 20:51:40.393228 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:40.393233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:40.393325 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:40.393458 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:40.394084 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:40.395222 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:40.396736 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:40.397415 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:40.397557 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:40.398793 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:40.398804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:40.398809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:40.398832 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:40.398886 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:40.398908 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:40.399009 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:40.399123 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 20:51:40.399218 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:51:40.400873 27695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37721 } }
I20260926 20:51:40.401408 27695 leader_election.cc:302] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fcb07d7a2554e4c88001e5e8899b6a0; no voters:
I20260926 20:51:40.402002 27695 leader_election.cc:288] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:40.402140 27705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:40.403571 27705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [term 1 LEADER]: Becoming Leader. State: Replica: 8fcb07d7a2554e4c88001e5e8899b6a0, State: Running, Role: LEADER
I20260926 20:51:40.404129 27705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37721 } }
I20260926 20:51:40.404379 27695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:40.407013 27707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fcb07d7a2554e4c88001e5e8899b6a0. Latest consensus state: current_term: 1 leader_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37721 } } }
I20260926 20:51:40.407346 27707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:40.407856 27706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fcb07d7a2554e4c88001e5e8899b6a0" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 37721 } } }
I20260926 20:51:40.408176 27706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:40.408257 27713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:40.413820 27713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:40.424275 27713 catalog_manager.cc:1347] Generated new cluster ID: 1b58ea559e064a19b2bb68166624f29d
I20260926 20:51:40.424430 27713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:40.442327 27713 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:40.444267 27713 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:40.452490 27713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fcb07d7a2554e4c88001e5e8899b6a0: Generated new TSK 0
I20260926 20:51:40.453616 27713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:40.470067 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27628
W20260926 20:51:40.471369 27690 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47653
W20260926 20:51:40.472180 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27628: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:40.472359 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27628
2026-09-26T20:51:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:40 dist-test-slave-z7j8 krb5kdc[27726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:40 dist-test-slave-z7j8 krb5kdc[27726](info): set up 2 sockets
Sep 26 20:51:40 dist-test-slave-z7j8 krb5kdc[27726](info): commencing operation
krb5kdc: starting...
W20260926 20:51:41.587146 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:41 dist-test-slave-z7j8 krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:41.854864 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:40047
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:40047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:42.196005 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:42.196321 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:42.196414 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:42.203868 27739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:42.204000 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:42.204052 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:42.204108 27739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:42.204162 27739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:42.214942 27739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:40047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:40047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:42.216262 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:42.217543 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:42.230562 27744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:42.230527 27745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:42.230584 27747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:42.231329 27739 server_base.cc:1034] running on GCE node
I20260926 20:51:42.232179 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:42.233659 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:42.234929 27739 hybrid_clock.cc:648] HybridClock initialized: now 1790455902234867 us; error 57 us; skew 500 ppm
I20260926 20:51:42.235344 27739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:42 dist-test-slave-z7j8 krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455902, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:42.243894 27739 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:42.245676 27739 webserver.cc:460] Webserver started at http://127.5.78.62:36993/ using document root <none> and password file <none>
I20260926 20:51:42.246539 27739 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:42.246702 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:42.247041 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:42.251515 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "9c2d80f5d6b84e18b410cb4987a60cbf"
format_stamp: "Formatted at 2026-09-26 20:51:42 on dist-test-slave-z7j8"
server_key: "21d24282c60ae78935231ffcde732715"
server_key_iv: "1252578e261afef40aac7b96b894f928"
server_key_version: "encryptionkey@0"
I20260926 20:51:42.252380 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "9c2d80f5d6b84e18b410cb4987a60cbf"
format_stamp: "Formatted at 2026-09-26 20:51:42 on dist-test-slave-z7j8"
server_key: "21d24282c60ae78935231ffcde732715"
server_key_iv: "1252578e261afef40aac7b96b894f928"
server_key_version: "encryptionkey@0"
I20260926 20:51:42.258173 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:51:42.262336 27754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:42.263985 27739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:51:42.264155 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "9c2d80f5d6b84e18b410cb4987a60cbf"
format_stamp: "Formatted at 2026-09-26 20:51:42 on dist-test-slave-z7j8"
server_key: "21d24282c60ae78935231ffcde732715"
server_key_iv: "1252578e261afef40aac7b96b894f928"
server_key_version: "encryptionkey@0"
I20260926 20:51:42.264344 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:42.270864 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:42.275432 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:42.275708 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:42.295400 27739 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:40047
I20260926 20:51:42.295495 27805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:40047 every 8 connection(s)
I20260926 20:51:42.296919 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:42.301008 27806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:42.301424 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27739
I20260926 20:51:42.301612 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:42.302040 5432 external_mini_cluster.cc:1428] Setting key 0bf868a8ec20cda31f0935d6f4590d3f
I20260926 20:51:42.311326 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:42.303858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38029 (local address 127.5.78.62:40047)
0926 20:51:42.304821 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:42.304834 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:42.304878 (+ 44us) server_negotiation.cc:407] Connection header received
0926 20:51:42.307370 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:42.307397 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:42.307871 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:42.308266 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:42.309497 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:42.309504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:42.309519 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:42.309540 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:42.309611 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:42.309637 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:42.309824 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:42.309997 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 20:51:42.310237 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":409,"threads_started":1}
I20260926 20:51:42.315496 27806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf: Bootstrap starting.
I20260926 20:51:42.315711 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:42.318799 27806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:42.319875 27806 log.cc:824] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:42.323371 27806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf: No bootstrap required, opened a new log
I20260926 20:51:42.324116 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:42.321321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51585 (local address 127.5.78.62:40047)
0926 20:51:42.321547 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:42.321558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:42.322033 (+ 475us) server_negotiation.cc:407] Connection header received
0926 20:51:42.322220 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:42.322224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:42.322273 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:42.322358 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:42.323350 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:42.323358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:42.323361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:51:42.323376 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:42.323412 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:51:42.323438 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:42.323578 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:42.323768 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 20:51:42.323837 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:51:42.331415 27806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40047 } }
I20260926 20:51:42.331847 27806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:51:42.331964 27806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2d80f5d6b84e18b410cb4987a60cbf, State: Initialized, Role: FOLLOWER
I20260926 20:51:42.332614 27806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40047 } }
I20260926 20:51:42.332890 27806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:51:42.333046 27806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:51:42.333243 27806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:42.335878 27806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40047 } }
I20260926 20:51:42.336390 27806 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c2d80f5d6b84e18b410cb4987a60cbf; no voters:
I20260926 20:51:42.337028 27806 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:51:42.337195 27816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:42.337896 27816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [term 1 LEADER]: Becoming Leader. State: Replica: 9c2d80f5d6b84e18b410cb4987a60cbf, State: Running, Role: LEADER
I20260926 20:51:42.338475 27816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40047 } }
I20260926 20:51:42.339311 27806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:42.341353 27817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40047 } } }
I20260926 20:51:42.341785 27817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:42.342362 27818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2d80f5d6b84e18b410cb4987a60cbf. Latest consensus state: current_term: 1 leader_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2d80f5d6b84e18b410cb4987a60cbf" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 40047 } } }
I20260926 20:51:42.343163 27818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:42.343765 27821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:42.349597 27821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:42.359357 27821 catalog_manager.cc:1347] Generated new cluster ID: 5c2e1b03a6204b96aed53c7febc747ee
I20260926 20:51:42.359501 27821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:42.375809 27821 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:42.376258 27821 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:42.384140 27821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c2d80f5d6b84e18b410cb4987a60cbf: Generated new TSK 0
I20260926 20:51:42.385071 27821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:42.395709 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27739
W20260926 20:51:42.397415 27801 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38029
W20260926 20:51:42.398181 5432 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27739: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:51:42.398350 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27739
2026-09-26T20:51:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1925 ms)
[----------] 18 tests from AuthTokenIssuingTest (35596 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:42 dist-test-slave-z7j8 krb5kdc[27837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:42 dist-test-slave-z7j8 krb5kdc[27837](info): set up 2 sockets
Sep 26 20:51:42 dist-test-slave-z7j8 krb5kdc[27837](info): commencing operation
krb5kdc: starting...
W20260926 20:51:43.527051 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.008s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:43 dist-test-slave-z7j8 krb5kdc[27837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:43.793938 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33367
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33367,127.5.78.61:39283,127.5.78.60:46493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:44.131624 27850 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:51:44.131946 27850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:44.132017 27850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:44.132128 27850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:44.139351 27850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:44.139441 27850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:44.139497 27850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:44.139552 27850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:44.139614 27850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:44.149446 27850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33367,127.5.78.61:39283,127.5.78.60:46493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:33367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:44.150696 27850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:44.151949 27850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:44.163437 27855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:44.163502 27858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:44.163470 27856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:44.164266 27850 server_base.cc:1034] running on GCE node
I20260926 20:51:44.165035 27850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:44.166498 27850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:44.167750 27850 hybrid_clock.cc:648] HybridClock initialized: now 1790455904167663 us; error 57 us; skew 500 ppm
I20260926 20:51:44.168246 27850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:44 dist-test-slave-z7j8 krb5kdc[27837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:44.171733 27850 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:44.173461 27850 webserver.cc:460] Webserver started at http://127.5.78.62:39501/ using document root <none> and password file <none>
I20260926 20:51:44.174222 27850 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:44.174355 27850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:44.174772 27850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:44.177984 27850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "38be408a1a994bf6b48c16ddfdb53d83"
format_stamp: "Formatted at 2026-09-26 20:51:44 on dist-test-slave-z7j8"
server_key: "dca4d5b41ff5af43842ec7eb89705882"
server_key_iv: "98abf9cf40f68ddb007f628d448be3a6"
server_key_version: "encryptionkey@0"
I20260926 20:51:44.178953 27850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "38be408a1a994bf6b48c16ddfdb53d83"
format_stamp: "Formatted at 2026-09-26 20:51:44 on dist-test-slave-z7j8"
server_key: "dca4d5b41ff5af43842ec7eb89705882"
server_key_iv: "98abf9cf40f68ddb007f628d448be3a6"
server_key_version: "encryptionkey@0"
I20260926 20:51:44.184768 27850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:51:44.188493 27865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:44.190045 27850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:51:44.190248 27850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "38be408a1a994bf6b48c16ddfdb53d83"
format_stamp: "Formatted at 2026-09-26 20:51:44 on dist-test-slave-z7j8"
server_key: "dca4d5b41ff5af43842ec7eb89705882"
server_key_iv: "98abf9cf40f68ddb007f628d448be3a6"
server_key_version: "encryptionkey@0"
I20260926 20:51:44.190508 27850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:44.204636 27850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:44.209621 27850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:44.209923 27850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:44.229444 27850 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:33367
I20260926 20:51:44.229470 27916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:33367 every 8 connection(s)
I20260926 20:51:44.230980 27850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:44.234910 27917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:44.241170 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27850
I20260926 20:51:44.241403 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:44.242233 5432 external_mini_cluster.cc:1428] Setting key f68eff9e35df8569ae04edc1a35a72a8
I20260926 20:51:44.242211 27917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:44 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.61@KRBTEST.COM; defaulting to no policy
I20260926 20:51:44.271230 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.246317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33367 (local address 127.0.0.1:35666)
0926 20:51:44.247298 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.247351 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:44.250271 (+ 2920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:44.251383 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:44.251393 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:44.252149 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:44.252504 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.252522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.254994 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.254998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.256172 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.256186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.257563 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.257568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.257715 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.258537 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:44.258561 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:44.261254 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:44.266548 (+ 5294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.266561 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.266572 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.267054 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.267631 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.267635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.267638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.267827 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.268975 (+ 1148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:44.268984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:44.269272 (+ 288us) client_negotiation.cc:769] Sending connection context
0926 20:51:44.269513 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 20:51:44.269963 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":364,"threads_started":1}
I20260926 20:51:44.271265 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.247563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.5.78.62:33367)
0926 20:51:44.248199 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:44.248205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:44.248251 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:51:44.250780 (+ 2529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:44.250787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:44.251092 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:44.251290 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:44.252769 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.254804 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.256336 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.257328 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.257531 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.261674 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:44.261689 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:44.261711 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:44.261763 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:44.266273 (+ 4510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.267223 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.267236 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.267242 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.267466 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.267975 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.267981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.267985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.268781 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:44.269019 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:44.269736 (+ 717us) server_negotiation.cc:299] Negotiation successful
0926 20:51:44.270196 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":464,"thread_start_us":306,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.61@KRBTEST.COM" created.
I20260926 20:51:44.274829 27917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:44.276093 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.275653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39780)
0926 20:51:44.275847 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.275976 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260926 20:51:44.276399 27868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
I20260926 20:51:44.278118 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.277698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39782)
0926 20:51:44.277869 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.277968 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 20:51:44.278326 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39283: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab.
Entry for principal kudu/127.5.78.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab.
I20260926 20:51:44.305806 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:39283
--webserver_interface=127.5.78.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab
--principal=kudu/127.5.78.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33367,127.5.78.61:39283,127.5.78.60:46493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:51:44.327772 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } attempt: 1
I20260926 20:51:44.329110 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.328517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39796)
0926 20:51:44.328777 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.328904 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 20:51:44.330132 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.329747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39806)
0926 20:51:44.329903 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.329984 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:51:44.330329 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39283: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
I20260926 20:51:44.398609 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } attempt: 2
I20260926 20:51:44.399964 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.399412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39808)
0926 20:51:44.399681 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.399792 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 20:51:44.401091 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.400682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39816)
0926 20:51:44.400839 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.400935 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:51:44.401362 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39283: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
I20260926 20:51:44.492662 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } attempt: 3
I20260926 20:51:44.493914 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.493381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39822)
0926 20:51:44.493622 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.493744 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 20:51:44.495330 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.494790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39838)
0926 20:51:44.494986 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.495125 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 20:51:44.495602 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39283: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
I20260926 20:51:44.638903 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } attempt: 4
W20260926 20:51:44.639837 27924 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260926 20:51:44.640175 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.639555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39852)
0926 20:51:44.639908 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.640021 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
W20260926 20:51:44.640307 27924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:44.640436 27924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:44.640580 27924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 20:51:44.641243 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.640850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39854)
0926 20:51:44.641007 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.641102 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 20:51:44.641470 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39283: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39283: connect: Connection refused (error 111)
W20260926 20:51:44.648520 27924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:44.648629 27924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:44.648687 27924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:44.648739 27924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:44.648801 27924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:44.659009 27924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33367,127.5.78.61:39283,127.5.78.60:46493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab
--principal=kudu/127.5.78.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:39283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.78.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:44.660414 27924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:44.661680 27924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:44.674197 27929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:44.674237 27930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:44.674197 27932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:44.674981 27924 server_base.cc:1034] running on GCE node
I20260926 20:51:44.675773 27924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:44.677258 27924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:44.678586 27924 hybrid_clock.cc:648] HybridClock initialized: now 1790455904678514 us; error 40 us; skew 500 ppm
I20260926 20:51:44.678962 27924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:44 dist-test-slave-z7j8 krb5kdc[27837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:44.682386 27924 init.cc:377] Logged in from keytab as kudu/127.5.78.61@KRBTEST.COM (short username kudu)
I20260926 20:51:44.684055 27924 webserver.cc:460] Webserver started at http://127.5.78.61:38139/ using document root <none> and password file <none>
I20260926 20:51:44.684804 27924 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:44.684949 27924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:44.685297 27924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:44.688512 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data/instance:
uuid: "223d78b6e3f0490db78ba907934634af"
format_stamp: "Formatted at 2026-09-26 20:51:44 on dist-test-slave-z7j8"
server_key: "e4892a1bd16c45701a0ff42ad0cd87d9"
server_key_iv: "7aa6fe2e7eaf47cf15ed89c733c35558"
server_key_version: "encryptionkey@0"
I20260926 20:51:44.689355 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal/instance:
uuid: "223d78b6e3f0490db78ba907934634af"
format_stamp: "Formatted at 2026-09-26 20:51:44 on dist-test-slave-z7j8"
server_key: "e4892a1bd16c45701a0ff42ad0cd87d9"
server_key_iv: "7aa6fe2e7eaf47cf15ed89c733c35558"
server_key_version: "encryptionkey@0"
I20260926 20:51:44.695013 27924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:51:44.698571 27939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:44.699898 27924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:44.700081 27924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal
uuid: "223d78b6e3f0490db78ba907934634af"
format_stamp: "Formatted at 2026-09-26 20:51:44 on dist-test-slave-z7j8"
server_key: "e4892a1bd16c45701a0ff42ad0cd87d9"
server_key_iv: "7aa6fe2e7eaf47cf15ed89c733c35558"
server_key_version: "encryptionkey@0"
I20260926 20:51:44.700316 27924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:44.719655 27924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:44.723716 27924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:44.723990 27924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:44.742086 27924 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.61:39283
I20260926 20:51:44.742098 27990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.61:39283 every 8 connection(s)
I20260926 20:51:44.743464 27924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
I20260926 20:51:44.747172 27991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:44.751641 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27924
I20260926 20:51:44.751853 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-1/wal/instance
I20260926 20:51:44.752328 5432 external_mini_cluster.cc:1428] Setting key cea30031fb466f5a3025de00fae7adf3
I20260926 20:51:44.753744 27991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:44 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:44.776962 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.756394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.5.78.62:33367)
0926 20:51:44.756656 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:44.756665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:44.757753 (+ 1088us) server_negotiation.cc:407] Connection header received
0926 20:51:44.759727 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:44.759737 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:44.759845 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:44.759973 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:44.761922 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.763205 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.764810 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.765608 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.765771 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.769653 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:44.769662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:44.769670 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:44.769718 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:44.773442 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.774342 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.774350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.774356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.774544 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.774957 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.774963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.774969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.775435 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:44.775591 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:44.776318 (+ 727us) server_negotiation.cc:299] Negotiation successful
0926 20:51:44.776574 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:51:44.777724 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.756747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33367 (local address 127.0.0.1:35680)
0926 20:51:44.757561 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.757612 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:44.759427 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:44.760291 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:44.760305 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:44.761082 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:44.761715 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.761733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.763401 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.763405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.764622 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.764636 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.765834 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.765839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.766050 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.766825 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:44.766849 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:44.769351 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:44.773640 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.773649 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.773673 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.774178 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.774661 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.774665 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.774667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.774842 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.775566 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:44.775572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:44.775930 (+ 358us) client_negotiation.cc:769] Sending connection context
0926 20:51:44.776175 (+ 245us) client_negotiation.cc:240] Negotiation successful
0926 20:51:44.776569 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":362,"threads_started":1}
WARNING: no policy specified for kudu/127.5.78.60@KRBTEST.COM; defaulting to no policy
I20260926 20:51:44.779597 27991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.60@KRBTEST.COM" created.
Sep 26 20:51:44 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:51:44.794865 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.780406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39868)
0926 20:51:44.780573 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.780594 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:44.780702 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:44.781871 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:44.781876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:44.782327 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:44.782530 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.782540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.783965 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.783972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.784837 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.784848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.785712 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.785718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.785912 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.786617 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:44.786634 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:44.788947 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:44.792287 (+ 3340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.792294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.792298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.792657 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.793032 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.793036 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.793040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.793188 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.793796 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:44.793801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:44.793970 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 20:51:44.794131 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 20:51:44.794476 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 20:51:44.794977 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.780697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.5.78.61:39283)
0926 20:51:44.781451 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:44.781457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:44.781482 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:51:44.781526 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:44.781532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:44.781722 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:44.781884 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:44.782694 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.783813 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.784976 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.785551 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.785692 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.789173 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:44.789185 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:44.789200 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:44.789234 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:44.792133 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.792785 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.792795 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.792800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.792918 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.793299 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.793306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.793310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.793673 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:44.793898 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:44.794286 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 20:51:44.794592 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":578,"thread_start_us":419,"threads_started":1}
I20260926 20:51:44.797499 27991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:44.798852 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.798227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45600)
0926 20:51:44.798466 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.798633 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 20:51:44.799175 27942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
I20260926 20:51:44.800872 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.800404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45610)
0926 20:51:44.800603 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.800715 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 20:51:44.801182 27991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab.
Entry for principal kudu/127.5.78.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab.
I20260926 20:51:44.814921 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:46493
--webserver_interface=127.5.78.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab
--principal=kudu/127.5.78.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:44595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:33367,127.5.78.61:39283,127.5.78.60:46493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:51:44.850544 27991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 1
I20260926 20:51:44.851825 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.851294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45618)
0926 20:51:44.851517 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.851631 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 20:51:44.852984 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.852548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45630)
0926 20:51:44.852739 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.852840 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 20:51:44.853266 27991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
I20260926 20:51:44.921537 27991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 2
I20260926 20:51:44.922910 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.922355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45634)
0926 20:51:44.922610 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.922736 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260926 20:51:44.924142 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.923708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45650)
0926 20:51:44.923896 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.923970 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 20:51:44.924355 27991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
I20260926 20:51:44.940765 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } attempt: 5
Sep 26 20:51:44 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:51:44.954953 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.941748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.5.78.61:39283)
0926 20:51:44.941915 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:44.941919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:44.941938 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:51:44.942010 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:44.942014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:44.942075 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:44.942184 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:44.943175 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.943853 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.945573 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.946037 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.946139 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.949858 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:44.949863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:44.949866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:44.949901 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:44.952439 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.953068 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.953075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.953079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.953203 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:44.953573 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:44.953579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:44.953583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:44.953860 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:44.954014 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:44.954386 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:51:44.954634 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:51:44.954959 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.941487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39880)
0926 20:51:44.941721 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.941744 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:44.941880 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:44.942247 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:44.942252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:44.942790 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:44.943016 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.943028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.943999 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.944004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.945400 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:44.945414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:44.946218 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:44.946224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:44.946397 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:44.947101 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:44.947133 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:44.949633 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:44.952575 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.952581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.952585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.952947 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.953320 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:44.953325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:44.953329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:44.953466 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:44.953978 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:44.953983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:44.954136 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 20:51:44.954279 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 20:51:44.954589 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 20:51:44.956025 27917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:44.956979 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.956609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45652)
0926 20:51:44.956762 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.956853 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 20:51:44.957875 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:44.957573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45668)
0926 20:51:44.957704 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:44.957779 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 20:51:44.958061 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
I20260926 20:51:45.009284 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 1
I20260926 20:51:45.010486 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.009950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45674)
0926 20:51:45.010150 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.010264 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 20:51:45.011495 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.011177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45676)
0926 20:51:45.011320 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.011397 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 20:51:45.011713 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
I20260926 20:51:45.015570 27991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 3
I20260926 20:51:45.016597 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.016166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45680)
0926 20:51:45.016380 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.016460 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260926 20:51:45.017714 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.017318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45688)
0926 20:51:45.017539 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.017611 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260926 20:51:45.018002 27991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
I20260926 20:51:45.079949 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 2
I20260926 20:51:45.081076 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.080617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45690)
0926 20:51:45.080816 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.080923 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 20:51:45.082048 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.081722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45702)
0926 20:51:45.081867 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.081941 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 20:51:45.082240 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
W20260926 20:51:45.158197 27998 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:51:45.158686 27998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:45.158823 27998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:45.158978 27998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 20:51:45.161250 27991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 4
I20260926 20:51:45.162549 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.161980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45706)
0926 20:51:45.162225 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.162324 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:51:45.163558 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.163231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45716)
0926 20:51:45.163382 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.163458 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 20:51:45.163822 27991 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
W20260926 20:51:45.167366 27998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:45.167514 27998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:45.167647 27998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:45.167753 27998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:45.167857 27998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:45.178371 27998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:44595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:33367,127.5.78.61:39283,127.5.78.60:46493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab
--principal=kudu/127.5.78.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:46493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.78.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:45.179880 27998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:45.181303 27998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 20:51:45.188457 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 3
I20260926 20:51:45.189635 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.189039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45732)
0926 20:51:45.189300 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.189413 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 20:51:45.190789 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.190353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45746)
0926 20:51:45.190557 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.190647 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260926 20:51:45.191042 27917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:46493: Network error: Client connection negotiation failed: client connection to 127.5.78.60:46493: connect: Connection refused (error 111)
W20260926 20:51:45.194475 28003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:45.194658 28004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:45.195057 28006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:45.195272 27998 server_base.cc:1034] running on GCE node
I20260926 20:51:45.196273 27998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:45.197817 27998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:45.199118 27998 hybrid_clock.cc:648] HybridClock initialized: now 1790455905199021 us; error 41 us; skew 500 ppm
I20260926 20:51:45.199489 27998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455905, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:45.202910 27998 init.cc:377] Logged in from keytab as kudu/127.5.78.60@KRBTEST.COM (short username kudu)
I20260926 20:51:45.204627 27998 webserver.cc:460] Webserver started at http://127.5.78.60:34837/ using document root <none> and password file <none>
I20260926 20:51:45.205358 27998 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:45.205502 27998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:45.205855 27998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:45.209090 27998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data/instance:
uuid: "9f90da0b8a174421be64a47dca127fcf"
format_stamp: "Formatted at 2026-09-26 20:51:45 on dist-test-slave-z7j8"
server_key: "a76b706b3b04f4796c62f7a667d16147"
server_key_iv: "8823867b28c3cd6973f06c7a7816c09e"
server_key_version: "encryptionkey@0"
I20260926 20:51:45.210019 27998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal/instance:
uuid: "9f90da0b8a174421be64a47dca127fcf"
format_stamp: "Formatted at 2026-09-26 20:51:45 on dist-test-slave-z7j8"
server_key: "a76b706b3b04f4796c62f7a667d16147"
server_key_iv: "8823867b28c3cd6973f06c7a7816c09e"
server_key_version: "encryptionkey@0"
I20260926 20:51:45.215652 27998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:51:45.219491 28013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:45.220896 27998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:51:45.221098 27998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal
uuid: "9f90da0b8a174421be64a47dca127fcf"
format_stamp: "Formatted at 2026-09-26 20:51:45 on dist-test-slave-z7j8"
server_key: "a76b706b3b04f4796c62f7a667d16147"
server_key_iv: "8823867b28c3cd6973f06c7a7816c09e"
server_key_version: "encryptionkey@0"
I20260926 20:51:45.221334 27998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:45.240890 27998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:45.245083 27998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:45.245410 27998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:45.264544 27998 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.60:46493
I20260926 20:51:45.264566 28064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.60:46493 every 8 connection(s)
I20260926 20:51:45.265951 27998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
I20260926 20:51:45.269634 28065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:45.270658 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 27998
I20260926 20:51:45.270879 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455541382686-5432-0/minicluster-data/master-2/wal/instance
I20260926 20:51:45.271365 5432 external_mini_cluster.cc:1428] Setting key 8d415a41112ede534648dd8c4dfb4b6d
I20260926 20:51:45.279129 28065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455905, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:45.296314 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.273179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.5.78.60:46493)
0926 20:51:45.273977 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.273989 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.274032 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:51:45.275731 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.275754 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.276132 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.276472 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.278558 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.279914 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.281875 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.283126 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.283925 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.288062 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.288070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.288082 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.288118 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.291616 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.292445 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.292451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.292459 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.292608 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.293024 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.293031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.293039 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.293533 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.293842 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.294302 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.294875 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":358,"threads_started":1}
I20260926 20:51:45.297945 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.282124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.5.78.62:33367)
0926 20:51:45.282721 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.282731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.283253 (+ 522us) server_negotiation.cc:407] Connection header received
0926 20:51:45.283479 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.283486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.283567 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.283699 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.285314 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.286072 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.287662 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.288110 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.288277 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.291923 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.291930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.291935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.291977 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.295058 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.295842 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.295846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.295848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.295968 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.296377 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.296380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.296382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.296700 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.296855 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.297323 (+ 468us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.297535 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":331,"threads_started":1}
I20260926 20:51:45.297966 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.282142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:33367 (local address 127.0.0.1:35682)
0926 20:51:45.283078 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.283118 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:45.283341 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:45.283762 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:45.283769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:45.284734 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:45.285111 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.285126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.286249 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.286256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.287504 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.287518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.288343 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.288352 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.288561 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.289311 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:45.289331 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:45.291729 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:45.295180 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.295191 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.295197 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.295729 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.296074 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.296078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.296082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.296252 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.296829 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:45.296842 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:45.297019 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 20:51:45.297191 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 20:51:45.297503 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":572,"threads_started":1}
I20260926 20:51:45.299271 28065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455905, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:45.312289 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.299964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39283 (local address 127.0.0.1:39894)
0926 20:51:45.300122 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.300139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:45.300223 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:45.300564 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:45.300566 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:45.300870 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:45.301019 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.301026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.302124 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.302129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.303159 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.303170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.303998 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.304006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.304236 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.304854 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:45.304880 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:45.306846 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:45.309880 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.309885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.309889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.310246 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.310665 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.310669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.310673 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.310829 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.311365 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:45.311370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:45.311510 (+ 140us) client_negotiation.cc:769] Sending connection context
0926 20:51:45.311651 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 20:51:45.311932 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 20:51:45.312316 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.300069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39894 (local address 127.5.78.61:39283)
0926 20:51:45.300269 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.300275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.300293 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:51:45.300353 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.300358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.300433 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.300534 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.301160 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.301981 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.303309 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.303851 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.303945 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.307042 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.307047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.307050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.307095 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.309732 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.310365 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.310372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.310376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.310546 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.310963 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.310967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.310971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.311246 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.311400 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.311749 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.311946 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:51:45.313158 28065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:45.314035 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.299417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.5.78.62:33367)
0926 20:51:45.299570 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.299576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.299659 (+ 83us) server_negotiation.cc:407] Connection header received
0926 20:51:45.299847 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.299852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.299955 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.300046 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.301523 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.302250 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.303525 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.304067 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.304194 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.308460 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.308465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.308468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.308522 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.311189 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.311878 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.311883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.311887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.312003 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.312567 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.312572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.312576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.312871 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.313142 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.313478 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.313705 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455905, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:51:45.327406 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.313849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45756)
0926 20:51:45.314015 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.314032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:45.314160 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:45.314613 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:45.314618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:45.315096 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:45.315292 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.315304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.316384 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.316388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.317231 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.317242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.318468 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.318487 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.318813 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.319461 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:45.319477 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:45.321358 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:45.324329 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.324336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.324340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.324825 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.325250 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.325254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.325256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.325454 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.326305 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:45.326309 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:45.326505 (+ 196us) client_negotiation.cc:769] Sending connection context
0926 20:51:45.326663 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 20:51:45.326987 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 20:51:45.327476 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.314140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.5.78.60:46493)
0926 20:51:45.314304 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.314313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.314331 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:51:45.314373 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.314377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.314469 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.314568 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.315458 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.316272 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.317384 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.318147 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.318311 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.321527 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.321532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.321537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.321575 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.324145 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.324960 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.324964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.324966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.325141 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.325581 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.325587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.325593 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.325985 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.326144 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.326769 (+ 625us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.327197 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":171,"server-negotiator.queue_time_us":62}
I20260926 20:51:45.332687 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.315978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.5.78.61:39283)
0926 20:51:45.316138 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.316141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.316589 (+ 448us) server_negotiation.cc:407] Connection header received
0926 20:51:45.316800 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.316805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.316866 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.316974 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.318595 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.319710 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.321016 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.321459 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.321610 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.326255 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.326261 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.326266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.326308 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.329782 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.330737 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.330741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.330743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.330873 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.331390 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.331394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.331396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.331654 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.331880 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.332252 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.332426 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 20:51:45.334395 28065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: Bootstrap starting.
I20260926 20:51:45.338368 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:45.338809 28065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:45.340299 28065 log.cc:824] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:45.343451 28065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: No bootstrap required, opened a new log
I20260926 20:51:45.342078 27892 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "38be408a1a994bf6b48c16ddfdb53d83"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35696
I20260926 20:51:45.351150 28065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.351550 28065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f90da0b8a174421be64a47dca127fcf, State: Initialized, Role: FOLLOWER
I20260926 20:51:45.352048 28065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.353118 28072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.353510 28072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:45.354200 28065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:51:45.364300 28083 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:51:45.364446 28083 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:51:45.368270 27917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 4
I20260926 20:51:45.362123 27998 master_runner.cc:186] Error getting master registration for 127.5.78.62:33367: OK, instance_id { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" instance_seqno: 1790455904223756 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:51:45.371754 27998 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
I20260926 20:51:45.385319 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.369063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45758)
0926 20:51:45.369290 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.369315 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:45.369462 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:45.369931 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:45.369938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:45.370494 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:45.370785 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.370802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.372308 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.372317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.373838 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.373855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.375003 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.375010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.375225 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.375960 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:45.375989 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:45.378897 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:45.382182 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.382187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.382192 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.382662 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.383161 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.383165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.383168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.383361 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.384235 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:45.384242 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:45.384443 (+ 201us) client_negotiation.cc:769] Sending connection context
0926 20:51:45.384592 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 20:51:45.384923 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 20:51:45.385850 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.369365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.5.78.60:46493)
0926 20:51:45.369572 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.369578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.369599 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:51:45.369661 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.369668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.369750 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.369857 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.370986 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.372134 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.374029 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.374814 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.374933 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.379140 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.379146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.379149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.379183 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.382016 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.382807 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.382813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.382822 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.383011 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.383513 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.383518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.383521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.384054 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.384231 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.384787 (+ 556us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.385172 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:51:45.390146 27917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83: Bootstrap starting.
I20260926 20:51:45.394342 27917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:45.395689 27917 log.cc:824] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:45.398504 27917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83: No bootstrap required, opened a new log
I20260926 20:51:45.405805 27917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.406196 27917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38be408a1a994bf6b48c16ddfdb53d83, State: Initialized, Role: FOLLOWER
I20260926 20:51:45.406806 27917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.407833 28086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.408209 28086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:45.409045 27917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:45.409250 27892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 20:51:45.409528 27892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:45.413316 27892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.416488 27892 leader_election.cc:288] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [CANDIDATE]: Term 1 election: Requested vote from peers 223d78b6e3f0490db78ba907934634af (127.5.78.61:39283), 9f90da0b8a174421be64a47dca127fcf (127.5.78.60:46493)
I20260926 20:51:45.416908 28040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38be408a1a994bf6b48c16ddfdb53d83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9f90da0b8a174421be64a47dca127fcf"
I20260926 20:51:45.417514 28040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:45.422353 28040 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38be408a1a994bf6b48c16ddfdb53d83 in term 1.
I20260926 20:51:45.423623 27868 leader_election.cc:302] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38be408a1a994bf6b48c16ddfdb53d83, 9f90da0b8a174421be64a47dca127fcf; no voters:
I20260926 20:51:45.424232 28086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:45.434544 28086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [term 1 LEADER]: Becoming Leader. State: Replica: 38be408a1a994bf6b48c16ddfdb53d83, State: Running, Role: LEADER
I20260926 20:51:45.435979 28086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.452739 28095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38be408a1a994bf6b48c16ddfdb53d83. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.453315 28095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: This master's current role is: LEADER
W20260926 20:51:45.462917 27871 tablet.cc:2369] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:51:45.463019 27991 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } attempt: 5
I20260926 20:51:45.477948 28104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:45.494856 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.452887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.5.78.61:39283)
0926 20:51:45.453113 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.453121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.459158 (+ 6037us) server_negotiation.cc:407] Connection header received
0926 20:51:45.459422 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.459432 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.459556 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.459686 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.464737 (+ 5051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.466266 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.470800 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.472001 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.472145 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.484453 (+ 12308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.484459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.484464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.484514 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.488223 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.491934 (+ 3711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.491940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.491945 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.492103 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.492556 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.492561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.492564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.492921 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.493171 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.494152 (+ 981us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.494431 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260926 20:51:45.497648 28019 tablet.cc:2369] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:51:45.485867 27850 master_runner.cc:186] Error getting master registration for 127.5.78.61:39283: OK, instance_id { permanent_uuid: "223d78b6e3f0490db78ba907934634af" instance_seqno: 1790455904736848 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:51:45.499511 28104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:51:45.499879 27850 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 20:51:45 dist-test-slave-z7j8 krb5kdc[27837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455904, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
I20260926 20:51:45.510197 28040 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [term 1 FOLLOWER]: Refusing update from remote peer 38be408a1a994bf6b48c16ddfdb53d83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:45.511374 28086 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:45.517128 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.460710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.5.78.60:46493)
0926 20:51:45.460965 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.460975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.472875 (+ 11900us) server_negotiation.cc:407] Connection header received
0926 20:51:45.474146 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.474155 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.474294 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.474398 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.483834 (+ 9436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.485047 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.496204 (+ 11157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.496944 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.497056 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.500601 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.500611 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.500618 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.500666 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.504440 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.509341 (+ 4901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.509348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.509384 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.509664 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.510543 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.510547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.510550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.511262 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.511699 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.512709 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.513129 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 20:51:45.540201 28110 mvcc.cc:205] Tried to move back new op lower bound from 7333707388956401664 to 7333707388681162752. Current Snapshot: MvccSnapshot[applied={T|T < 7333707388956401664}]
I20260926 20:51:45.541452 28072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38be408a1a994bf6b48c16ddfdb53d83. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.541857 28072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:45.543429 28111 mvcc.cc:205] Tried to move back new op lower bound from 7333707388956401664 to 7333707388681162752. Current Snapshot: MvccSnapshot[applied={T|T < 7333707388956401664}]
I20260926 20:51:45.554358 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.463701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:46493 (local address 127.0.0.1:45780)
0926 20:51:45.464182 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:45.464203 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:45.464317 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:45.479056 (+ 14739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:45.479066 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:45.479704 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:45.479996 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.480012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.482802 (+ 2790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.482808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.484214 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:45.484229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.501944 (+ 17715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.501954 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:45.502182 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.504704 (+ 2522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:45.504733 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:45.507466 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:45.549021 (+ 41555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.549028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.549033 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.549712 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.551677 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:45.551683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:45.551687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:45.551901 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:45.553187 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:45.553193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:45.553380 (+ 187us) client_negotiation.cc:769] Sending connection context
0926 20:51:45.553542 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 20:51:45.553944 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352}
I20260926 20:51:45.554745 28072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.555058 28072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:45.554981 28086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.555312 28086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:45.559911 28108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.464543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.5.78.60:46493)
0926 20:51:45.478645 (+ 14102us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.478651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.478680 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:51:45.478752 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.478763 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.478842 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.478980 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.480346 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.482618 (+ 2272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.488569 (+ 5951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.489292 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.520235 (+ 30943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.520543 (+ 308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.520548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.520553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.520589 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.532093 (+ 11504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.550007 (+ 17914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.550014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.550021 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.550236 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.552091 (+ 1855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.552097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.552100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.552500 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.555593 (+ 3093us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.555735 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.559099 (+ 3364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13945,"thread_start_us":14671,"threads_started":1}
I20260926 20:51:45.566381 27991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: Bootstrap starting.
I20260926 20:51:45.571611 28104 catalog_manager.cc:1347] Generated new cluster ID: 1068d0b110c04447b0edf6aeff063d08
I20260926 20:51:45.571758 28104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:45.577111 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:45.456223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.5.78.62:33367)
0926 20:51:45.456458 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.456468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.459852 (+ 3384us) server_negotiation.cc:407] Connection header received
0926 20:51:45.461775 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.461784 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.461905 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.462008 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.520985 (+ 58977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.522187 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.527892 (+ 5705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:45.528647 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:45.528780 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:45.556018 (+ 27238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:45.556025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:45.556030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:45.556077 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:45.569218 (+ 13141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.571382 (+ 2164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.571390 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.571395 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.571751 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:45.572832 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:45.572841 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:45.572878 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:45.573781 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:45.574352 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:45.575673 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0926 20:51:45.576225 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 20:51:45.591116 27991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:45.594632 27991 log.cc:824] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:45.606176 27991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: No bootstrap required, opened a new log
I20260926 20:51:45.623195 27991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.623801 27991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223d78b6e3f0490db78ba907934634af, State: Initialized, Role: FOLLOWER
I20260926 20:51:45.624490 27991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } }
I20260926 20:51:45.627228 28104 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:45.629200 28104 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:45.638999 28116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.639602 28116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:45.645907 27965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:45.649648 27965 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [term 1 FOLLOWER]: Refusing update from remote peer 38be408a1a994bf6b48c16ddfdb53d83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:45.650332 27966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38be408a1a994bf6b48c16ddfdb53d83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "223d78b6e3f0490db78ba907934634af"
I20260926 20:51:45.652083 28104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83: Generated new TSK 0
I20260926 20:51:45.653157 28104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:45.653715 27966 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38be408a1a994bf6b48c16ddfdb53d83 in term 1.
I20260926 20:51:45.656421 28095 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:45.659245 27991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:45.713828 28116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38be408a1a994bf6b48c16ddfdb53d83. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.714337 28116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:45.714828 28122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:45.715401 28122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af [sys.catalog]: This master's current role is: FOLLOWER
W20260926 20:51:45.775002 28069 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.765152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.5.78.62:33367)
0926 20:51:45.765405 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.765412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.771891 (+ 6479us) server_negotiation.cc:407] Connection header received
0926 20:51:45.773458 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.773465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.773548 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.773617 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.774751 (+ 1134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35712: BlockingRecv error: recv got EOF from 127.0.0.1:35712 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 20:51:45.776847 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.761377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.5.78.61:39283)
0926 20:51:45.761573 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.761579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.771372 (+ 9793us) server_negotiation.cc:407] Connection header received
0926 20:51:45.775405 (+ 4033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.775411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.775524 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.775594 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.776608 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39938: BlockingRecv error: recv got EOF from 127.0.0.1:39938 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 20:51:45.779620 28141 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 20:51:45.780409 28108 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.772648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.5.78.60:46493)
0926 20:51:45.772821 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.772826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.778113 (+ 5287us) server_negotiation.cc:407] Connection header received
0926 20:51:45.779120 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.779126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.779215 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.779289 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.780171 (+ 882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45784: BlockingRecv error: recv got EOF from 127.0.0.1:45784 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 20:51:45.798902 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.796564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.5.78.61:39283)
0926 20:51:45.796751 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.796757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.797210 (+ 453us) server_negotiation.cc:407] Connection header received
0926 20:51:45.797447 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.797453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.797583 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.797704 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.798573 (+ 869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39954: BlockingRecv error: recv got EOF from 127.0.0.1:39954 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 20:51:45.812469 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.808469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.5.78.61:39283)
0926 20:51:45.808703 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.808712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.811079 (+ 2367us) server_negotiation.cc:407] Connection header received
0926 20:51:45.811309 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.811314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.811400 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.811511 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.812258 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39970: BlockingRecv error: recv got EOF from 127.0.0.1:39970 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 20:51:45.828512 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.826218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39984 (local address 127.5.78.61:39283)
0926 20:51:45.826502 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.826511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.827021 (+ 510us) server_negotiation.cc:407] Connection header received
0926 20:51:45.827257 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.827262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.827361 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.827482 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.828224 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39984: BlockingRecv error: recv got EOF from 127.0.0.1:39984 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260926 20:51:45.842850 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.840336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40000 (local address 127.5.78.61:39283)
0926 20:51:45.840620 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.840629 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.841161 (+ 532us) server_negotiation.cc:407] Connection header received
0926 20:51:45.841399 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.841405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.841493 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.841603 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.842611 (+ 1008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40000: BlockingRecv error: recv got EOF from 127.0.0.1:40000 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260926 20:51:45.866850 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.864529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.5.78.61:39283)
0926 20:51:45.864771 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.864781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.865351 (+ 570us) server_negotiation.cc:407] Connection header received
0926 20:51:45.865586 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.865592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.865679 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.865791 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.866581 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40016: BlockingRecv error: recv got EOF from 127.0.0.1:40016 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 20:51:45.888607 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.886396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.5.78.61:39283)
0926 20:51:45.886675 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.886685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.887199 (+ 514us) server_negotiation.cc:407] Connection header received
0926 20:51:45.887439 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.887444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.887516 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.887624 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.888366 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40026: BlockingRecv error: recv got EOF from 127.0.0.1:40026 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260926 20:51:45.926453 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.924021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.5.78.61:39283)
0926 20:51:45.924279 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.924289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.924873 (+ 584us) server_negotiation.cc:407] Connection header received
0926 20:51:45.925121 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.925126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.925236 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.925347 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.926171 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40030: BlockingRecv error: recv got EOF from 127.0.0.1:40030 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 20:51:45.998205 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:45.995988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40044 (local address 127.5.78.61:39283)
0926 20:51:45.996244 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:45.996253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:45.996822 (+ 569us) server_negotiation.cc:407] Connection header received
0926 20:51:45.997055 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:45.997060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:45.997134 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:45.997244 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:45.997985 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40044: BlockingRecv error: recv got EOF from 127.0.0.1:40044 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 20:51:46.131214 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:46.129346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.5.78.61:39283)
0926 20:51:46.129603 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.129609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.129852 (+ 243us) server_negotiation.cc:407] Connection header received
0926 20:51:46.130015 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.130019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.130094 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.130184 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:46.130911 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40060: BlockingRecv error: recv got EOF from 127.0.0.1:40060 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 20:51:46.195227 28086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38be408a1a994bf6b48c16ddfdb53d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38be408a1a994bf6b48c16ddfdb53d83" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 33367 } } peers { permanent_uuid: "223d78b6e3f0490db78ba907934634af" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39283 } } peers { permanent_uuid: "9f90da0b8a174421be64a47dca127fcf" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 46493 } } }
I20260926 20:51:46.195597 28086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38be408a1a994bf6b48c16ddfdb53d83 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:46.366732 28083 catalog_manager.cc:1259] Loaded cluster ID: 1068d0b110c04447b0edf6aeff063d08
I20260926 20:51:46.366895 28083 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: loading cluster ID for follower catalog manager: success
I20260926 20:51:46.369714 28083 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: acquiring CA information for follower catalog manager: success
I20260926 20:51:46.371815 28083 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f90da0b8a174421be64a47dca127fcf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 20:51:46.393283 27995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:46.391500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.5.78.61:39283)
0926 20:51:46.391822 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.391831 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.391968 (+ 137us) server_negotiation.cc:407] Connection header received
0926 20:51:46.392125 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.392130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.392204 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.392306 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:46.393029 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40070: BlockingRecv error: recv got EOF from 127.0.0.1:40070 (error 108)
Metrics: {"server-negotiator.queue_time_us":178}
I20260926 20:51:46.783991 28141 catalog_manager.cc:1259] Loaded cluster ID: 1068d0b110c04447b0edf6aeff063d08
I20260926 20:51:46.784111 28141 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: loading cluster ID for follower catalog manager: success
I20260926 20:51:46.786852 28141 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: acquiring CA information for follower catalog manager: success
I20260926 20:51:46.788789 28141 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 223d78b6e3f0490db78ba907934634af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:51:46.918890 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:46.911488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.5.78.61:39283)
0926 20:51:46.912315 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.912319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.912337 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:51:46.912418 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.912422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.912573 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.912687 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:46.913292 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.914094 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.916217 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.916655 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.916771 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:46.917431 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:46.918061 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:46.918179 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:46.918374 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 20:51:46.918515 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":555,"threads_started":1}
I20260926 20:51:46.931914 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:46.925626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.5.78.60:46493)
0926 20:51:46.926691 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.926695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.926711 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:51:46.926806 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.926810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.926964 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.927145 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:46.927570 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.928274 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.929603 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.930030 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.930171 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:46.930745 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:46.931286 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:46.931473 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:46.931526 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 20:51:46.931613 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":872,"threads_started":1}
I20260926 20:51:46.956364 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:46.943769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40084 (local address 127.5.78.61:39283)
0926 20:51:46.943993 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.944001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.948769 (+ 4768us) server_negotiation.cc:407] Connection header received
0926 20:51:46.949044 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.949051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.949144 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.949217 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:46.950114 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.951315 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.953632 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.954458 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.954567 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:46.955344 (+ 777us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:46.955506 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:46.955575 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:46.955911 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 20:51:46.956011 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:51:46.958889 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:46.943388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.5.78.62:33367)
0926 20:51:46.948263 (+ 4875us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.948268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.948288 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:51:46.948389 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.948393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.948586 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.948789 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:46.949846 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.951427 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.954601 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.955559 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.956379 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:46.956676 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:46.957344 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:46.958232 (+ 888us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:46.958298 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 20:51:46.958445 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4764,"thread_start_us":4675,"threads_started":1}
I20260926 20:51:46.964102 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:46.949572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.5.78.60:46493)
0926 20:51:46.949786 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:46.949795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:46.957434 (+ 7639us) server_negotiation.cc:407] Connection header received
0926 20:51:46.957758 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:46.957765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:46.957874 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:46.957944 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:46.958585 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.959774 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.961892 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:46.962619 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:46.962757 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:46.963264 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:46.963408 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:46.963506 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:46.963676 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 20:51:46.963776 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:51:46.964733 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27850
I20260926 20:51:46.965866 27912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:47.122105 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27850
I20260926 20:51:47.143653 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27924
I20260926 20:51:47.144815 27986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:47.262980 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27924
I20260926 20:51:47.282472 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 27998
I20260926 20:51:47.283658 28060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:47.390497 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 27998
2026-09-26T20:51:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4998 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:51:47 dist-test-slave-z7j8 krb5kdc[28221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:51:47 dist-test-slave-z7j8 krb5kdc[28221](info): set up 2 sockets
Sep 26 20:51:47 dist-test-slave-z7j8 krb5kdc[28221](info): commencing operation
krb5kdc: starting...
W20260926 20:51:48.510973 5432 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.007s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/test-user.keytab.
Sep 26 20:51:48 dist-test-slave-z7j8 krb5kdc[28221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.78.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
Entry for principal kudu/127.5.78.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab.
I20260926 20:51:48.783571 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44055
--webserver_interface=127.5.78.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44055,127.5.78.61:39075,127.5.78.60:36081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:51:49.126626 28234 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:51:49.127019 28234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:49.127152 28234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:49.127295 28234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:49.134858 28234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:49.134981 28234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:49.135109 28234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:49.135196 28234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:49.135303 28234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:49.145076 28234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44055,127.5.78.61:39075,127.5.78.60:36081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.62.keytab
--principal=kudu/127.5.78.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.62:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.78.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:49.146468 28234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:49.147852 28234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:49.160135 28239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:49.160180 28242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:49.160084 28240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:49.161154 28234 server_base.cc:1034] running on GCE node
I20260926 20:51:49.162025 28234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:49.163621 28234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:49.164924 28234 hybrid_clock.cc:648] HybridClock initialized: now 1790455909164854 us; error 45 us; skew 500 ppm
I20260926 20:51:49.165316 28234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:49 dist-test-slave-z7j8 krb5kdc[28221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:49.168773 28234 init.cc:377] Logged in from keytab as kudu/127.5.78.62@KRBTEST.COM (short username kudu)
I20260926 20:51:49.170393 28234 webserver.cc:460] Webserver started at http://127.5.78.62:43229/ using document root <none> and password file <none>
I20260926 20:51:49.171170 28234 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:49.171325 28234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:49.171669 28234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:49.174764 28234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data/instance:
uuid: "053e70a2369944738deed84cbdca2969"
format_stamp: "Formatted at 2026-09-26 20:51:49 on dist-test-slave-z7j8"
server_key: "2393621621afd48c73225486463bf9bc"
server_key_iv: "5fe949a7b724976a54d3b271b30c4ac6"
server_key_version: "encryptionkey@0"
I20260926 20:51:49.175544 28234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal/instance:
uuid: "053e70a2369944738deed84cbdca2969"
format_stamp: "Formatted at 2026-09-26 20:51:49 on dist-test-slave-z7j8"
server_key: "2393621621afd48c73225486463bf9bc"
server_key_iv: "5fe949a7b724976a54d3b271b30c4ac6"
server_key_version: "encryptionkey@0"
I20260926 20:51:49.180884 28234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 20:51:49.184499 28249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:49.185905 28234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:51:49.186098 28234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal
uuid: "053e70a2369944738deed84cbdca2969"
format_stamp: "Formatted at 2026-09-26 20:51:49 on dist-test-slave-z7j8"
server_key: "2393621621afd48c73225486463bf9bc"
server_key_iv: "5fe949a7b724976a54d3b271b30c4ac6"
server_key_version: "encryptionkey@0"
I20260926 20:51:49.186332 28234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:49.203508 28234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:49.207774 28234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:49.208067 28234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:49.226668 28234 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.62:44055
I20260926 20:51:49.226688 28300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.62:44055 every 8 connection(s)
I20260926 20:51:49.228034 28234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/data/info.pb
I20260926 20:51:49.229892 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 28234
I20260926 20:51:49.230086 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-0/wal/instance
I20260926 20:51:49.230651 5432 external_mini_cluster.cc:1428] Setting key 09b9483c0b85fea659087eac6c11d396
I20260926 20:51:49.232524 28301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:49.242146 28301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.78.61@KRBTEST.COM; defaulting to no policy
Sep 26 20:51:49 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.61@KRBTEST.COM" created.
I20260926 20:51:49.264839 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.245235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.5.78.62:44055)
0926 20:51:49.246904 (+ 1669us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:49.246914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:49.246962 (+ 48us) server_negotiation.cc:407] Connection header received
0926 20:51:49.248494 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:49.248499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:49.248742 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:49.248950 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:49.250214 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.251464 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.252922 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.253462 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.253664 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.257187 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:49.257198 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:49.257211 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:49.257244 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:49.260443 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.261399 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.261407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.261409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.261559 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.262001 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.262004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.262006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.262610 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:49.262765 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:49.263454 (+ 689us) server_negotiation.cc:299] Negotiation successful
0926 20:51:49.263782 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":1451,"thread_start_us":1251,"threads_started":1}
I20260926 20:51:49.264873 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.245234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44055 (local address 127.0.0.1:54236)
0926 20:51:49.246050 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.246096 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:49.248113 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:49.248958 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:49.248966 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:49.249678 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:49.250003 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.250021 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.251620 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.251626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.252787 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.252799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.253707 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.253715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.253885 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.254568 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:49.254590 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:49.256839 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:49.260606 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.260622 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.260639 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.261257 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.261677 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.261684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.261690 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.261895 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.262747 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:49.262758 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:49.263039 (+ 281us) client_negotiation.cc:769] Sending connection context
0926 20:51:49.263328 (+ 289us) client_negotiation.cc:240] Negotiation successful
0926 20:51:49.263977 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":90,"thread_start_us":355,"threads_started":1}
I20260926 20:51:49.268265 28301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:49.269402 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.269047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52096)
0926 20:51:49.269202 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.269300 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 20:51:49.269704 28252 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
I20260926 20:51:49.271256 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.270909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52106)
0926 20:51:49.271049 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.271119 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 20:51:49.271488 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39075: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab.
Entry for principal kudu/127.5.78.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab.
I20260926 20:51:49.294052 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:39075
--webserver_interface=127.5.78.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab
--principal=kudu/127.5.78.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44055,127.5.78.61:39075,127.5.78.60:36081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:51:49.320851 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } attempt: 1
I20260926 20:51:49.322075 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.321627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52116)
0926 20:51:49.321835 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.321946 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 20:51:49.323154 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.322827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52124)
0926 20:51:49.322959 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.323030 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 20:51:49.323381 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39075: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
I20260926 20:51:49.391662 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } attempt: 2
I20260926 20:51:49.392983 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.392449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52136)
0926 20:51:49.392731 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.392834 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260926 20:51:49.394083 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.393722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52150)
0926 20:51:49.393872 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.393961 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 20:51:49.394299 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39075: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
I20260926 20:51:49.485608 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } attempt: 3
I20260926 20:51:49.486883 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.486355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52162)
0926 20:51:49.486632 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.486734 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260926 20:51:49.488008 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.487621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52166)
0926 20:51:49.487773 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.487860 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 20:51:49.488296 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39075: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
I20260926 20:51:49.631582 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } attempt: 4
I20260926 20:51:49.632905 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.632332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52176)
0926 20:51:49.632674 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.632775 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260926 20:51:49.634024 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.633658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52188)
0926 20:51:49.633823 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.633907 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 20:51:49.634312 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.61:39075: Network error: Client connection negotiation failed: client connection to 127.5.78.61:39075: connect: Connection refused (error 111)
W20260926 20:51:49.636773 28308 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:51:49.637084 28308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:49.637163 28308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:49.637238 28308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:49.644799 28308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:49.644906 28308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:49.644964 28308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:49.645007 28308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:49.645071 28308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:49.655370 28308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44055,127.5.78.61:39075,127.5.78.60:36081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.61.keytab
--principal=kudu/127.5.78.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.61:39075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.78.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:49.656716 28308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:49.658041 28308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:51:49.670783 28313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:49.670826 28314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:49.670852 28316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:49.671545 28308 server_base.cc:1034] running on GCE node
I20260926 20:51:49.672356 28308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:49.673812 28308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:49.675036 28308 hybrid_clock.cc:648] HybridClock initialized: now 1790455909674962 us; error 49 us; skew 500 ppm
I20260926 20:51:49.675415 28308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:49 dist-test-slave-z7j8 krb5kdc[28221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:49.678740 28308 init.cc:377] Logged in from keytab as kudu/127.5.78.61@KRBTEST.COM (short username kudu)
I20260926 20:51:49.680294 28308 webserver.cc:460] Webserver started at http://127.5.78.61:36011/ using document root <none> and password file <none>
I20260926 20:51:49.681084 28308 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:49.681226 28308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:49.681572 28308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:49.684706 28308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data/instance:
uuid: "89665242e1fd4842974085808331f9e9"
format_stamp: "Formatted at 2026-09-26 20:51:49 on dist-test-slave-z7j8"
server_key: "6384cd52f84897f97e63ef393f572a5f"
server_key_iv: "9535b7833d571d137882cff722820e3b"
server_key_version: "encryptionkey@0"
I20260926 20:51:49.685520 28308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal/instance:
uuid: "89665242e1fd4842974085808331f9e9"
format_stamp: "Formatted at 2026-09-26 20:51:49 on dist-test-slave-z7j8"
server_key: "6384cd52f84897f97e63ef393f572a5f"
server_key_iv: "9535b7833d571d137882cff722820e3b"
server_key_version: "encryptionkey@0"
I20260926 20:51:49.691179 28308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:51:49.694927 28323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:49.696480 28308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:51:49.696700 28308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal
uuid: "89665242e1fd4842974085808331f9e9"
format_stamp: "Formatted at 2026-09-26 20:51:49 on dist-test-slave-z7j8"
server_key: "6384cd52f84897f97e63ef393f572a5f"
server_key_iv: "9535b7833d571d137882cff722820e3b"
server_key_version: "encryptionkey@0"
I20260926 20:51:49.696947 28308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:49.717696 28308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:49.722481 28308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:49.722797 28308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:49.742734 28308 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.61:39075
I20260926 20:51:49.742758 28374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.61:39075 every 8 connection(s)
I20260926 20:51:49.744205 28308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/data/info.pb
I20260926 20:51:49.747822 28375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:49.750324 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 28308
I20260926 20:51:49.750536 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-1/wal/instance
I20260926 20:51:49.751046 5432 external_mini_cluster.cc:1428] Setting key 49aee778d262bdd35449c513157d0075
I20260926 20:51:49.756870 28375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:49 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.78.60@KRBTEST.COM; defaulting to no policy
I20260926 20:51:49.780550 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.760398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.5.78.62:44055)
0926 20:51:49.760639 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:49.760645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:49.761733 (+ 1088us) server_negotiation.cc:407] Connection header received
0926 20:51:49.763704 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:49.763713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:49.763809 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:49.763934 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:49.765504 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.766678 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.768405 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.769199 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.769359 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.773504 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:49.773514 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:49.773522 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:49.773574 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:49.777442 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.778324 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.778334 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.778340 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.778516 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.778922 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.778925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.778927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.779208 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:49.779385 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:49.780121 (+ 736us) server_negotiation.cc:299] Negotiation successful
0926 20:51:49.780313 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.78.60@KRBTEST.COM" created.
I20260926 20:51:49.781837 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.760751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44055 (local address 127.0.0.1:54244)
0926 20:51:49.761532 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.761588 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:49.763415 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:49.764181 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:49.764189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:49.764757 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:49.765311 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.765329 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.766884 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.766889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.768222 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.768238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.769458 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.769464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.769699 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.770575 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:49.770601 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:49.773218 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:49.777634 (+ 4416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.777642 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.777658 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.778160 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.778631 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.778635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.778637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.778821 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.779422 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:49.779432 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:49.779792 (+ 360us) client_negotiation.cc:769] Sending connection context
0926 20:51:49.780034 (+ 242us) client_negotiation.cc:240] Negotiation successful
0926 20:51:49.780566 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":351,"threads_started":1}
I20260926 20:51:49.783703 28375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:49 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:51:49.800196 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.784479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52196)
0926 20:51:49.784682 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.784702 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:49.784806 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:49.785604 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:49.785615 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:49.786099 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:49.786325 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.786339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.787743 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.787751 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.789270 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.789288 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.790508 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.790512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.790689 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.791345 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:49.791361 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:49.793526 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:49.797807 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.797810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.797813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.798153 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.798579 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.798582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.798585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.798717 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.799466 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:49.799469 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:49.799574 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 20:51:49.799676 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 20:51:49.799881 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":33}
I20260926 20:51:49.800871 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.784574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.5.78.61:39075)
0926 20:51:49.785192 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:49.785198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:49.785230 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:51:49.785271 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:49.785276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:49.785449 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:49.785559 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:49.786545 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.787582 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.789440 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.790312 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.790491 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.793753 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:49.793768 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:49.793787 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:49.793829 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:49.797675 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.798281 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.798292 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.798299 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.798465 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.798842 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.798849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.798855 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.799332 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:49.799542 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:49.799878 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 20:51:49.800175 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":467,"thread_start_us":326,"threads_started":1}
I20260926 20:51:49.803725 28375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:49.805004 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.804545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54592)
0926 20:51:49.804743 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.804858 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260926 20:51:49.805301 28327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:49.806936 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.806605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54604)
0926 20:51:49.806750 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.806835 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 20:51:49.807171 28375 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.78.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab.
Entry for principal kudu/127.5.78.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab.
I20260926 20:51:49.818998 5432 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
/tmp/dist-test-taskb6fvOW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:36081
--webserver_interface=127.5.78.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab
--principal=kudu/127.5.78.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.78.20:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.78.62:44055,127.5.78.61:39075,127.5.78.60:36081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:51:49.856519 28375 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 1
I20260926 20:51:49.857765 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.857299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54616)
0926 20:51:49.857489 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.857584 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 20:51:49.858896 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.858522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54626)
0926 20:51:49.858709 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.858787 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 20:51:49.859120 28375 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:49.927350 28375 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 2
I20260926 20:51:49.928508 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.928039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54632)
0926 20:51:49.928241 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.928351 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 20:51:49.929628 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.929264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54646)
0926 20:51:49.929416 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.929495 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 20:51:49.929863 28375 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:49.933575 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } attempt: 5
Sep 26 20:51:49 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:51:49.946766 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.934524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.5.78.61:39075)
0926 20:51:49.934703 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:49.934709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:49.934731 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:51:49.934809 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:49.934821 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:49.934882 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:49.934987 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:49.935771 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.936466 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.937627 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.938052 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.938194 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.941535 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:49.941543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:49.941552 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:49.941587 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:49.944141 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.944818 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.944825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.944828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.944954 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:49.945321 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:49.945328 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:49.945331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:49.945644 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:49.945800 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:49.946184 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 20:51:49.946398 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:51:49.946762 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.934294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52202)
0926 20:51:49.934496 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.934518 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:49.934681 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:49.935020 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:49.935025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:49.935446 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:49.935615 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.935624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.936613 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.936618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.937472 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:49.937489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:49.938202 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:49.938208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:49.938398 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:49.939105 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:49.939123 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:49.941324 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:49.944280 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.944286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.944290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.944670 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.945066 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:49.945074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:49.945077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:49.945214 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:49.945766 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:49.945770 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:49.945930 (+ 160us) client_negotiation.cc:769] Sending connection context
0926 20:51:49.946074 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 20:51:49.946362 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 20:51:49.947984 28301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:49.949034 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.948595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54658)
0926 20:51:49.948756 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.948840 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 20:51:49.949887 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:49.949587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54660)
0926 20:51:49.949717 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:49.949804 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 20:51:49.950104 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:50.001335 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 1
I20260926 20:51:50.002699 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.002123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54662)
0926 20:51:50.002348 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.002521 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 20:51:50.003836 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.003479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54674)
0926 20:51:50.003656 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.003728 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 20:51:50.004067 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:50.021073 28375 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 3
I20260926 20:51:50.022202 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.021750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54682)
0926 20:51:50.021947 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.022042 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 20:51:50.023283 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.022893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54696)
0926 20:51:50.023115 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.023184 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260926 20:51:50.023478 28375 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:50.072278 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 2
I20260926 20:51:50.073555 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.073043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54706)
0926 20:51:50.073294 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.073387 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:51:50.074553 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.074196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54712)
0926 20:51:50.074332 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.074442 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 20:51:50.074780 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
W20260926 20:51:50.154983 28382 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:51:50.155367 28382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:51:50.155495 28382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:51:50.155642 28382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:51:50.162810 28382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:51:50.162935 28382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:51:50.163044 28382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:51:50.163146 28382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:51:50.163234 28382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:51:50.166673 28375 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 4
I20260926 20:51:50.167770 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.167273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54726)
0926 20:51:50.167505 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.167605 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 20:51:50.168725 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.168391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54728)
0926 20:51:50.168532 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.168625 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 20:51:50.168912 28375 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
I20260926 20:51:50.173185 28382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.78.20:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.78.62:44055,127.5.78.61:39075,127.5.78.60:36081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/krb5kdc/kudu_127.5.78.60.keytab
--principal=kudu/127.5.78.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.78.60:36081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.78.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:51:50.174513 28382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:51:50.175832 28382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 20:51:50.181008 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 3
I20260926 20:51:50.182628 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.181798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54742)
0926 20:51:50.182260 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.182374 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 20:51:50.183879 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.183416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54744)
0926 20:51:50.183640 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.183751 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260926 20:51:50.184111 28301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.78.60:36081: Network error: Client connection negotiation failed: client connection to 127.5.78.60:36081: connect: Connection refused (error 111)
W20260926 20:51:50.188488 28387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:50.188519 28388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:51:50.188778 28390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:51:50.189447 28382 server_base.cc:1034] running on GCE node
I20260926 20:51:50.190302 28382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:51:50.191910 28382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:51:50.193167 28382 hybrid_clock.cc:648] HybridClock initialized: now 1790455910193093 us; error 53 us; skew 500 ppm
I20260926 20:51:50.193526 28382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:51:50.196738 28382 init.cc:377] Logged in from keytab as kudu/127.5.78.60@KRBTEST.COM (short username kudu)
I20260926 20:51:50.198215 28382 webserver.cc:460] Webserver started at http://127.5.78.60:41163/ using document root <none> and password file <none>
I20260926 20:51:50.198946 28382 fs_manager.cc:359] Metadata directory not provided
I20260926 20:51:50.199033 28382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:51:50.199301 28382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:51:50.202518 28382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data/instance:
uuid: "853cb3004ef345598ff4618afb3c06f0"
format_stamp: "Formatted at 2026-09-26 20:51:50 on dist-test-slave-z7j8"
server_key: "7dd351235d47c995a91db45facac61bf"
server_key_iv: "b3705aaa849d3b24ab83e2925528f3ef"
server_key_version: "encryptionkey@0"
I20260926 20:51:50.203263 28382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal/instance:
uuid: "853cb3004ef345598ff4618afb3c06f0"
format_stamp: "Formatted at 2026-09-26 20:51:50 on dist-test-slave-z7j8"
server_key: "7dd351235d47c995a91db45facac61bf"
server_key_iv: "b3705aaa849d3b24ab83e2925528f3ef"
server_key_version: "encryptionkey@0"
I20260926 20:51:50.208232 28382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:51:50.211578 28397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:51:50.212896 28382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 20:51:50.213068 28382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data,/tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal
uuid: "853cb3004ef345598ff4618afb3c06f0"
format_stamp: "Formatted at 2026-09-26 20:51:50 on dist-test-slave-z7j8"
server_key: "7dd351235d47c995a91db45facac61bf"
server_key_iv: "b3705aaa849d3b24ab83e2925528f3ef"
server_key_version: "encryptionkey@0"
I20260926 20:51:50.213296 28382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:51:50.224637 28382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:51:50.229100 28382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:51:50.229395 28382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:51:50.247932 28382 rpc_server.cc:307] RPC server started. Bound to: 127.5.78.60:36081
I20260926 20:51:50.248014 28448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.78.60:36081 every 8 connection(s)
I20260926 20:51:50.249282 28382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/data/info.pb
I20260926 20:51:50.253051 28449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:51:50.254833 5432 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu as pid 28382
I20260926 20:51:50.255031 5432 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb6fvOW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455541382686-5432-0/minicluster-data/master-2/wal/instance
I20260926 20:51:50.255504 5432 external_mini_cluster.cc:1428] Setting key 57f97b09776de3bf83379e7586864b95
I20260926 20:51:50.262943 28449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:50.280154 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.257552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54756 (local address 127.5.78.60:36081)
0926 20:51:50.258473 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.258494 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.258553 (+ 59us) server_negotiation.cc:407] Connection header received
0926 20:51:50.260269 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.260301 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.260869 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.261295 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.262699 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.264736 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.266794 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.267884 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.268094 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.272578 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.272587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.272605 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.272653 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.275993 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.276728 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.276735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.276743 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.276892 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.277310 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.277315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.277317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.277858 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.278167 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.278671 (+ 504us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.279116 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":384,"threads_started":1}
I20260926 20:51:50.281570 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.266154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.5.78.62:44055)
0926 20:51:50.266375 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.266383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.267035 (+ 652us) server_negotiation.cc:407] Connection header received
0926 20:51:50.267235 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.267239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.267312 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.267397 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.268846 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.269548 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.271206 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.271940 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.272088 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.275979 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.275984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.275987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.276021 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.278666 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.279502 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.279508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.279511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.279659 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.280041 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.280052 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.280054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.280340 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.280490 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.280991 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.281214 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:51:50.281630 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.266068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.62:44055 (local address 127.0.0.1:54252)
0926 20:51:50.266866 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.266911 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:50.267098 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:50.267506 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:50.267514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:50.268428 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:50.268675 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.268684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.269717 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.269725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.271037 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.271050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.272173 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.272178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.272306 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.272980 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:50.272997 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:50.275774 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:50.278797 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.278809 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.278816 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.279364 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.279774 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.279779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.279783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.279931 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.280537 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:50.280545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:50.280699 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 20:51:50.280879 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 20:51:50.281212 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":384,"threads_started":1}
I20260926 20:51:50.283108 28449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.62@KRBTEST.COM
I20260926 20:51:50.297382 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.283877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.61:39075 (local address 127.0.0.1:52218)
0926 20:51:50.284130 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.284149 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:50.284274 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:50.284607 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:50.284611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:50.285004 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:50.285145 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.285153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.286169 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.286175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.287126 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.287137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.287839 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.287845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.288017 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.288646 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:50.288673 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:50.291100 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:50.294866 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.294870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.294873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.295228 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.295643 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.295646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.295648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.295802 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.296391 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:50.296396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:50.296570 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 20:51:50.296735 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 20:51:50.296990 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260926 20:51:50.297431 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.283904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.5.78.61:39075)
0926 20:51:50.284085 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.284093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.284236 (+ 143us) server_negotiation.cc:407] Connection header received
0926 20:51:50.284393 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.284398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.284473 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.284585 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.285303 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.286019 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.287276 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.287693 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.287829 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.291289 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.291299 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.291306 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.291367 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.294735 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.295351 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.295358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.295364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.295531 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.295910 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.295914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.295917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.296242 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.296356 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.296856 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.297088 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:51:50.298511 28449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:51:50.298803 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.284097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.5.78.62:44055)
0926 20:51:50.284291 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.284296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.284312 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:51:50.284365 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.284368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.284421 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.284524 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.285686 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.286448 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.287614 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.288035 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.288166 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.292690 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.292697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.292703 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.292754 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.295895 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.296694 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.296719 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.296724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.296869 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.297278 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.297283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.297287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.297669 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.297934 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.298235 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.298478 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455910, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.60@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.78.61@KRBTEST.COM
I20260926 20:51:50.311784 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.299539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.5.78.60:36081)
0926 20:51:50.299716 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.299721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.299740 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:51:50.299842 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.299848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.299921 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.299991 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.300819 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.301625 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.302868 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.303283 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.303411 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.306321 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.306328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.306333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.306371 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.309328 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.309928 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.309936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.309939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.310050 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.310380 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.310387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.310390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.310685 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.310824 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.311174 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.311398 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":74}
I20260926 20:51:50.311882 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.299431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54762)
0926 20:51:50.299585 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.299605 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:50.299728 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:50.300050 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:50.300055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:50.300540 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:50.300690 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.300698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.301740 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.301748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.302740 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.302751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.303413 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.303419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.303619 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.304183 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:50.304198 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:50.306115 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:50.309454 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.309461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.309465 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.309816 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.310149 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.310156 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.310159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.310281 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.310800 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:50.310804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:50.310941 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 20:51:50.311094 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 20:51:50.311543 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":149}
I20260926 20:51:50.314837 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.300444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.5.78.61:39075)
0926 20:51:50.300656 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.300662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.300678 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:51:50.300769 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.300773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.300841 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.300929 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.302130 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.302903 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.304094 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.304508 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.304642 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.309398 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.309403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.309406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.309436 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.312232 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.312894 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.312900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.312903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.313037 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.313424 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.313429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.313432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.313707 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.313987 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.314278 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.314527 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 20:51:50.317353 28449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: Bootstrap starting.
I20260926 20:51:50.319881 5432 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:51:50.320937 28449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:50.322101 28449 log.cc:824] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:50.324949 28449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: No bootstrap required, opened a new log
I20260926 20:51:50.322440 28276 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "053e70a2369944738deed84cbdca2969"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54266
I20260926 20:51:50.331966 28449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.332293 28449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 853cb3004ef345598ff4618afb3c06f0, State: Initialized, Role: FOLLOWER
I20260926 20:51:50.332717 28449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.333590 28455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.333938 28455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:50.334731 28449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:51:50.344620 28466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:51:50.344815 28466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:51:50.342129 28382 master_runner.cc:186] Error getting master registration for 127.5.78.62:44055: OK, instance_id { permanent_uuid: "053e70a2369944738deed84cbdca2969" instance_seqno: 1790455909221403 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:51:50.350777 28382 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:51:50.361326 28301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 4
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.62@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
I20260926 20:51:50.377321 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.362063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54766)
0926 20:51:50.362285 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.362308 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:50.362460 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:50.362969 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:50.362974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:50.363423 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:50.363607 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.363618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.364738 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.364746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.365738 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.365750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.366624 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.366630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.366869 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.367539 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:50.367560 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:50.370044 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:50.374272 (+ 4228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.374277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.374282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.374805 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.375324 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.375329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.375333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.375497 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.376247 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:50.376253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:50.376436 (+ 183us) client_negotiation.cc:769] Sending connection context
0926 20:51:50.376597 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 20:51:50.376905 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 20:51:50.377393 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.362115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54766 (local address 127.5.78.60:36081)
0926 20:51:50.362668 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.362672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.362691 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:51:50.362746 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.362750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.362811 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.362937 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.363776 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.364571 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.365904 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.366390 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.366613 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.370272 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.370279 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.370285 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.370330 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.374085 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.374948 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.374971 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.374976 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.375186 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.375625 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.375631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.375635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.376101 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.376301 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.376731 (+ 430us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.376970 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260926 20:51:50.382869 28301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969: Bootstrap starting.
I20260926 20:51:50.386019 28301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:50.387106 28301 log.cc:824] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:50.389807 28301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969: No bootstrap required, opened a new log
I20260926 20:51:50.395053 28301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.395358 28301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053e70a2369944738deed84cbdca2969, State: Initialized, Role: FOLLOWER
I20260926 20:51:50.395730 28301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.396631 28469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.396986 28469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:50.397732 28301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:50.397917 28276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 20:51:50.398166 28276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:50.401801 28276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.405259 28276 leader_election.cc:288] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [CANDIDATE]: Term 1 election: Requested vote from peers 89665242e1fd4842974085808331f9e9 (127.5.78.61:39075), 853cb3004ef345598ff4618afb3c06f0 (127.5.78.60:36081)
I20260926 20:51:50.407513 28424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "053e70a2369944738deed84cbdca2969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "853cb3004ef345598ff4618afb3c06f0"
I20260926 20:51:50.408041 28424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:50.411569 28424 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053e70a2369944738deed84cbdca2969 in term 1.
I20260926 20:51:50.412230 28253 leader_election.cc:302] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053e70a2369944738deed84cbdca2969, 853cb3004ef345598ff4618afb3c06f0; no voters:
I20260926 20:51:50.412739 28469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:51:50.418671 28469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [term 1 LEADER]: Becoming Leader. State: Replica: 053e70a2369944738deed84cbdca2969, State: Running, Role: LEADER
I20260926 20:51:50.419466 28469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.424276 28481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053e70a2369944738deed84cbdca2969. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.424690 28481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:50.426311 28486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:51:50.427220 28487 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:51:50.433465 28486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260926 20:51:50.463248 28255 tablet.cc:2369] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:51:50.432988 28234 master_runner.cc:186] Error getting master registration for 127.5.78.61:39075: OK, instance_id { permanent_uuid: "89665242e1fd4842974085808331f9e9" instance_seqno: 1790455909736934 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:51:50.468103 28375 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } attempt: 5
I20260926 20:51:50.472965 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.428703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.5.78.62:44055)
0926 20:51:50.428893 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.428901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.439610 (+ 10709us) server_negotiation.cc:407] Connection header received
0926 20:51:50.439983 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.439989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.440100 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.440184 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.444227 (+ 4043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.445480 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.449408 (+ 3928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.450495 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.450634 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.458026 (+ 7392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.458037 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.458046 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.458115 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.462803 (+ 4688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.465641 (+ 2838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.465652 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.465659 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.465997 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.466967 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.466973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.466977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.467783 (+ 806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.471462 (+ 3679us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.471787 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.472226 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:51:50.474462 28234 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:51:50.481940 28403 tablet.cc:2369] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:51:50.483873 28424 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [term 1 FOLLOWER]: Refusing update from remote peer 053e70a2369944738deed84cbdca2969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:50.484931 28481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:50.497216 28455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053e70a2369944738deed84cbdca2969. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.497568 28455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 20:51:50 dist-test-slave-z7j8 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455909, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.78.61@KRBTEST.COM for kudu/127.5.78.60@KRBTEST.COM
I20260926 20:51:50.504040 28490 mvcc.cc:205] Tried to move back new op lower bound from 7333707409326657536 to 7333707409085976576. Current Snapshot: MvccSnapshot[applied={T|T < 7333707409326657536}]
I20260926 20:51:50.505540 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.456919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.5.78.60:36081)
0926 20:51:50.457097 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.457103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.475851 (+ 18748us) server_negotiation.cc:407] Connection header received
0926 20:51:50.476302 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.476311 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.476410 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.476491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.479820 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.480896 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.486666 (+ 5770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.487536 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.487656 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.493390 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.493398 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.493404 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.493454 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.498043 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.501254 (+ 3211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.501261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.501266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.501474 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.502671 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.502678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.502682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.503151 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.503453 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.504751 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.505067 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:51:50.506274 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.447516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.5.78.61:39075)
0926 20:51:50.447734 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.447743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.463140 (+ 15397us) server_negotiation.cc:407] Connection header received
0926 20:51:50.465740 (+ 2600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.465746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.465848 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.465930 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.468552 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.469725 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.482683 (+ 12958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.483436 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.483553 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.497902 (+ 14349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.497912 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.497918 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.497978 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.502071 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.503109 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.503115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.503119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.503287 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.503768 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.503774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.503777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.504192 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.504456 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.505613 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.505876 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:51:50.511902 28494 mvcc.cc:205] Tried to move back new op lower bound from 7333707409326657536 to 7333707409085976576. Current Snapshot: MvccSnapshot[applied={T|T < 7333707409326657536}]
I20260926 20:51:50.514310 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.468735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.78.60:36081 (local address 127.0.0.1:54774)
0926 20:51:50.469007 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 20:51:50.469028 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:51:50.469142 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:51:50.492797 (+ 23655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:51:50.492804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:51:50.493364 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:51:50.493598 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.493611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.496277 (+ 2666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.496285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.497807 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:51:50.497827 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.498928 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.498937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:51:50.499148 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.499780 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:51:50.499805 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:51:50.502876 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:51:50.509122 (+ 6246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.509128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.509132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.509664 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.512035 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:51:50.512040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:51:50.512043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:51:50.512226 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:51:50.513086 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:51:50.513094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:51:50.513281 (+ 187us) client_negotiation.cc:769] Sending connection context
0926 20:51:50.513443 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 20:51:50.513814 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260926 20:51:50.515934 28493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:50.468791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.5.78.60:36081)
0926 20:51:50.491132 (+ 22341us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.491143 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.491172 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:51:50.491232 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.491239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.491353 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.493771 (+ 2418us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.494091 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.495450 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.498006 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:50.498755 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:50.499478 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:50.503280 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:51:50.503289 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:51:50.503294 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:51:50.503330 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:51:50.507993 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.511694 (+ 3701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.511704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.511709 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.511889 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:51:50.512384 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:51:50.512390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:51:50.512396 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:51:50.512939 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:51:50.514970 (+ 2031us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:50.515122 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 20:51:50.515394 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22171,"thread_start_us":22020,"threads_started":1}
I20260926 20:51:50.523177 28375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: Bootstrap starting.
I20260926 20:51:50.528321 28375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: Neither blocks nor log segments found. Creating new log.
I20260926 20:51:50.529992 28375 log.cc:824] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:51:50.532251 28455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.532584 28455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:50.543710 28469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.544041 28469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:50.545300 28486 catalog_manager.cc:1347] Generated new cluster ID: dfdcf3a4e8ba45619273e4158d016fe8
I20260926 20:51:50.545420 28486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:51:50.545801 28375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: No bootstrap required, opened a new log
I20260926 20:51:50.568919 28375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.569514 28375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89665242e1fd4842974085808331f9e9, State: Initialized, Role: FOLLOWER
I20260926 20:51:50.570173 28375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } }
I20260926 20:51:50.577059 28486 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:51:50.579140 28486 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:51:50.587067 28499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.588559 28375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:51:50.588617 28350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "053e70a2369944738deed84cbdca2969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "89665242e1fd4842974085808331f9e9"
I20260926 20:51:50.589155 28349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:51:50.598690 28349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [term 1 FOLLOWER]: Refusing update from remote peer 053e70a2369944738deed84cbdca2969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:51:50.594199 28499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:50.599964 28481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:51:50.602201 28350 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053e70a2369944738deed84cbdca2969 in term 1.
I20260926 20:51:50.606819 28486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969: Generated new TSK 0
I20260926 20:51:50.607903 28486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:51:50.647413 28499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053e70a2369944738deed84cbdca2969. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.647862 28499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:50.647921 28505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:50.648398 28505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:51:50.681249 28517 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 20:51:50.713245 28305 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.697166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.5.78.62:44055)
0926 20:51:50.697405 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.697413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.702889 (+ 5476us) server_negotiation.cc:407] Connection header received
0926 20:51:50.710895 (+ 8006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.710902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.711006 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.711137 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.712839 (+ 1702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54280: BlockingRecv error: recv got EOF from 127.0.0.1:54280 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 20:51:50.716054 28493 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.703012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.5.78.60:36081)
0926 20:51:50.703194 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.703201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.714160 (+ 10959us) server_negotiation.cc:407] Connection header received
0926 20:51:50.714336 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.714341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.714448 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.714551 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.715825 (+ 1274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54782: BlockingRecv error: recv got EOF from 127.0.0.1:54782 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260926 20:51:50.717147 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.697256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.5.78.61:39075)
0926 20:51:50.697424 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.697431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.714223 (+ 16792us) server_negotiation.cc:407] Connection header received
0926 20:51:50.715281 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.715291 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.715409 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.715511 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.716799 (+ 1288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52240: BlockingRecv error: recv got EOF from 127.0.0.1:52240 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 20:51:50.732646 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.722744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.5.78.61:39075)
0926 20:51:50.722925 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.722933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.731010 (+ 8077us) server_negotiation.cc:407] Connection header received
0926 20:51:50.731271 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.731277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.731429 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.731547 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.732393 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52256: BlockingRecv error: recv got EOF from 127.0.0.1:52256 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260926 20:51:50.740341 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.737217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.5.78.61:39075)
0926 20:51:50.737378 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.737384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.738920 (+ 1536us) server_negotiation.cc:407] Connection header received
0926 20:51:50.739127 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.739134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.739227 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.739341 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.740118 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52262: BlockingRecv error: recv got EOF from 127.0.0.1:52262 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 20:51:50.784926 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.776414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.5.78.61:39075)
0926 20:51:50.776602 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.776609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.783056 (+ 6447us) server_negotiation.cc:407] Connection header received
0926 20:51:50.783448 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.783455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.783568 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.783708 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.784625 (+ 917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52272: BlockingRecv error: recv got EOF from 127.0.0.1:52272 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 20:51:50.800983 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.792422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.5.78.61:39075)
0926 20:51:50.792645 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.792651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.799236 (+ 6585us) server_negotiation.cc:407] Connection header received
0926 20:51:50.799467 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.799474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.799592 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.799732 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.800701 (+ 969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52280: BlockingRecv error: recv got EOF from 127.0.0.1:52280 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260926 20:51:50.818168 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.814573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.5.78.61:39075)
0926 20:51:50.814776 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.814783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.816458 (+ 1675us) server_negotiation.cc:407] Connection header received
0926 20:51:50.816661 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.816669 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.816780 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.816914 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.817884 (+ 970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52288: BlockingRecv error: recv got EOF from 127.0.0.1:52288 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 20:51:50.841279 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.838673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.5.78.61:39075)
0926 20:51:50.838892 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.838899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.839536 (+ 637us) server_negotiation.cc:407] Connection header received
0926 20:51:50.839875 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.839881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.839989 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.840109 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.841020 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52290: BlockingRecv error: recv got EOF from 127.0.0.1:52290 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260926 20:51:50.886271 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.883711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.5.78.61:39075)
0926 20:51:50.883935 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.883943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.884556 (+ 613us) server_negotiation.cc:407] Connection header received
0926 20:51:50.884796 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.884802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.884905 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.885022 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.885984 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52306: BlockingRecv error: recv got EOF from 127.0.0.1:52306 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260926 20:51:50.964359 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:50.962512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.5.78.61:39075)
0926 20:51:50.962730 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:50.962736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:50.963020 (+ 284us) server_negotiation.cc:407] Connection header received
0926 20:51:50.963198 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:50.963204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:50.963301 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:50.963415 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:50.964118 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52316: BlockingRecv error: recv got EOF from 127.0.0.1:52316 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 20:51:51.097904 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:51.095795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52326 (local address 127.5.78.61:39075)
0926 20:51:51.096046 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.096052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.096377 (+ 325us) server_negotiation.cc:407] Connection header received
0926 20:51:51.096644 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.096651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.096767 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.096885 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:51.097697 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52326: BlockingRecv error: recv got EOF from 127.0.0.1:52326 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:51:51.147125 28481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "053e70a2369944738deed84cbdca2969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e70a2369944738deed84cbdca2969" member_type: VOTER last_known_addr { host: "127.5.78.62" port: 44055 } } peers { permanent_uuid: "89665242e1fd4842974085808331f9e9" member_type: VOTER last_known_addr { host: "127.5.78.61" port: 39075 } } peers { permanent_uuid: "853cb3004ef345598ff4618afb3c06f0" member_type: VOTER last_known_addr { host: "127.5.78.60" port: 36081 } } }
I20260926 20:51:51.147481 28481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053e70a2369944738deed84cbdca2969 [sys.catalog]: This master's current role is: LEADER
I20260926 20:51:51.347172 28466 catalog_manager.cc:1259] Loaded cluster ID: dfdcf3a4e8ba45619273e4158d016fe8
I20260926 20:51:51.347286 28466 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: loading cluster ID for follower catalog manager: success
I20260926 20:51:51.349972 28466 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: acquiring CA information for follower catalog manager: success
I20260926 20:51:51.352018 28466 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 853cb3004ef345598ff4618afb3c06f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 20:51:51.359254 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:51:51.357403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.5.78.61:39075)
0926 20:51:51.357577 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.357582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.357901 (+ 319us) server_negotiation.cc:407] Connection header received
0926 20:51:51.358070 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.358074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.358146 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.358243 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:51:51.359013 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52338: BlockingRecv error: recv got EOF from 127.0.0.1:52338 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:51:51.686123 28517 catalog_manager.cc:1259] Loaded cluster ID: dfdcf3a4e8ba45619273e4158d016fe8
I20260926 20:51:51.686290 28517 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: loading cluster ID for follower catalog manager: success
I20260926 20:51:51.689383 28517 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: acquiring CA information for follower catalog manager: success
I20260926 20:51:51.691671 28517 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89665242e1fd4842974085808331f9e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:51:51.883790 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:51.876145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52352 (local address 127.5.78.61:39075)
0926 20:51:51.876845 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.876851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.877146 (+ 295us) server_negotiation.cc:407] Connection header received
0926 20:51:51.877402 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.877406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.877554 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.877661 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:51.878224 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.879109 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.881144 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.881576 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.881679 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:51.882392 (+ 713us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:51.883039 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:51.883217 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:51.883316 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 20:51:51.883391 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":331,"threads_started":1}
I20260926 20:51:51.901284 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:51.891569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.5.78.60:36081)
0926 20:51:51.894818 (+ 3249us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.894823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.894842 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:51:51.894945 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.894949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.895095 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.895245 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:51.895845 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.896542 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.898663 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.899159 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.899271 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:51.899932 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:51.900606 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:51.900721 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:51.900861 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 20:51:51.900935 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3131,"thread_start_us":3023,"threads_started":1}
I20260926 20:51:51.933457 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:51.909150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.5.78.61:39075)
0926 20:51:51.909577 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.909585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.921934 (+ 12349us) server_negotiation.cc:407] Connection header received
0926 20:51:51.924347 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.924357 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.924506 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.924598 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:51.925685 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.927324 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.930998 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.931665 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.931767 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:51.932548 (+ 781us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:51.932689 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:51.932761 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:51.933034 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 20:51:51.933127 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260926 20:51:51.938172 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:51.909112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.5.78.60:36081)
0926 20:51:51.909388 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.909398 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.931074 (+ 21676us) server_negotiation.cc:407] Connection header received
0926 20:51:51.931374 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.931380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.931527 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.931600 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:51.932450 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.933492 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.935902 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.936610 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.936757 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:51.937220 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:51.937377 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:51.937490 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:51.937733 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 20:51:51.937815 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 20:51:51.942780 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:51:51.908835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54296 (local address 127.5.78.62:44055)
0926 20:51:51.924030 (+ 15195us) server_negotiation.cc:206] Beginning negotiation
0926 20:51:51.924040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:51:51.934365 (+ 10325us) server_negotiation.cc:407] Connection header received
0926 20:51:51.936524 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:51:51.936531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:51:51.936771 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:51:51.937012 (+ 241us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:51:51.937471 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.938723 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.940690 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:51:51.941109 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:51:51.941209 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:51:51.941728 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:51:51.942049 (+ 321us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:51:51.942128 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 20:51:51.942328 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 20:51:51.942397 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15005,"thread_start_us":15208,"threads_started":1}
I20260926 20:51:51.947806 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 28234
I20260926 20:51:51.948921 28296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:52.072824 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 28234
W20260926 20:51:52.093565 28399 connection.cc:535] server connection from 127.0.0.1:54766 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 20:51:52.093936 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 28308
I20260926 20:51:52.095166 28370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:52.201195 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 28308
I20260926 20:51:52.221925 5432 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu pid 28382
I20260926 20:51:52.223183 28444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:51:52.329838 5432 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb6fvOW/build/asan/bin/kudu with pid 28382
2026-09-26T20:51:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4935 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9933 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370909 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST